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ABSTRACT 

In this *tenth annual report on federally-funded ■ 
education programs, the authors give an overview of evaluation 
activities in the new Department of Education and describe changes In 
.the department's techniques and uses of evaluation*- The first chapter 
discusses the role of evaluation in ensuring that federal funds for 
education are speht_ efficiently, examines , the department's evaluation 
model and the dissemination of its results, ,and notes the 
introduction of time-saving techniques in assessing .program 
pvaluability, objectives, management, an* service delivery* flajor 
department evaluation activities at 'the elementary, secondary, and 
postsecondary levels and in special category programs are summarized 
in chapter two- The authors assess .the results of "^lementary and 
secondary programs for low-achieving students, point out changes made 
by programs tb enhance postsecondary 'educational* opportunity, „and 
list results ql categorical programs in vocational, adult, career, 
handicapped/ community, tfacher, Indian, and drug abuse education. 
The final chapter highlights findings on program legislation, - 
effectiveness, <ianage£nt, tpr content from 19 selected evaluation 
studies. (A^thor/RW) * , * * % 1 *> 
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Foreword 



This is the I(Jth annu^eporf to the Congress on federally funded Education 
prog*anr>s. These programs were t^jsferred in May 1980 to the. U.S. Department 
of Education from tte U.S. Office of Education'. This ypar's report, like its 
predecessors, responds to several congressional mandates. The original directive 
for the Annual Evaluation Report is contained in Section 417 (a) of the General 
Edu<^atiort Pro visions. Aft (GEPA).. The Report is to waluate the effectiveness of 
programs in achieving their legislated purposes and include recommendations for 
achieving greater effectiveness^ection 1246o$the Education Amendments pf 
1978 .(Public Law 95-561). mod if/ed ^he pnginal GEPA mandate to require that 
reporting on program effectiveness also indicate compliance with provisions of 
. the law on maintenance of non-Federal expenditures where such provisions exist. 
£ third requirement became law early in fiscal year (FY) 1981 as part of the 
Education Amendments o'f 1980 (Public Law 96-374). Section 130> further 
amends the GEPA mandate to require tabulations of available data to indicate % 
program and project impact by the sex, race, and age of beneficiaries. 

In addition to meeting these -mandated requirements, this regprt describes the 
^role of evaluation in improving education programs anctlhanges in evaluation 
activities in the new Department of Education (ED). It alio explains the 
evaluation mission in ED. ' 

Users of the previous, single-volume editions of this repoh will quickly note that 
this year's versionrhas been prepared and packaged differently. Past Office of 
Education (OE) reports were published in one volume. They.cootained overviews 
of evaluation activities with detailed pfogram material comprising the bulk of . 
the volume. The'Fy 1980 report has been divided irvto two volumes.^ The new,^ 
condensed volume 1 is intended for general .distribution. Detailed program-by- 
program Summaries ar6 n6w cqntained ivrx second volume which will be 
diftriButed to appropriate congressional committees and to other ^interested / 
reader^ upon request^ * ' " * 

Several reasons underlie this change in foynat. #he evaluatipn report is read -by 
a diverse audience with differing ne^ds and us,es for -evaluation Information. 
• Congressmen, State'and Jocal officials, the education community, parents, . " 
students, and concerned citizens all have a general interest in the effectiveness " 
of Federal education programs. .Specific information needs, however, differ* \ 
widely and detailed reporting is critical only in some instances. The two- volume 
report will enable ED to respond 'to' varying information requirements, depending 
upon the, needs of interested parties. The Department believes that the two- 
volume report wijl be a more flexible, efficient, and cost-effective way Jto 
disseminate evaluation information thaiS the large, single volume. 

Because* volume I is intended to provide £ broad overview of ED evaluatiorj 
activities, additional explanatory material has been included. This first volumd 
contains information on .the Department's innovative information f gatherinp and 
evaluative techniques, and on its management initiatives to better utilize, 
evaluation findings. Highlights of new evaluation findings and the uses of ED 
evaluations can be found ir> chapter 3. A general perspective on*major pD| 
program evalCation activities is contained inx&apter 2. Program areas covered 
include elementary and secondary education, postsecondary education, anc * 
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diverse special category programs. As previously mentioned, detailed >r 
information on ind*vkJual program evaluations which were in p^st annual 
evaluation reports h&s been updated for FY 1980 and-*ppears as volJrne II. Ml 
programs that were formerly the responsibility of the Office of Education are * 
covered in this year's report. The FY 19S1 report, representing the Department's 
first full fiscal year of oper-ation,' will contain evaluation Teports on these ^ 
programs as weH as comparable report^ on programs transferred to the 
Department from other Federal agencies as a result of the ED Organization \cx. 

* 

I would like to acknowledge the contributions of several individuals l& this year's 
Report. Dr. Thomas U hi man .translated the'idea of a first volume into reality by 
structuring and writing much of it. The Evaluation Coordination Staff, in 
particular Mr. Edward CJIaBsman, Ms* Jeane Onufry, and Ms. Elaine Green, hel^>e^ 
pull together and organize diverse evaluation materials from 3 variety of 
sources. Evaluation division directors and their staffs contributed specific 
program chapters and overview materials. Finally, Ms. Diana Carpenter's 
tireless typing was essential to the successful completion of this report. 

^ A 



, lohn Seal 

Acting Assistant Secretary ' 
for Management 
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On May 1980, the Department of Education (ED) became the 13th Cabirtet- 
level Department in tjie U.S. Government. Today, the Department's numerous 
programs focus on two primary goals. The first is to asSure all students, 
regardless of age, background, or native ability, equalaccess to the best possible 
education. The second goal is to he!p improve the quality of education for every 
student. 

One critical component of accountability in Federal education activities in the 
Office of Education (OE) and now in ED is evaluation. The legislation 
establishing the Department specifically cites improvements in evaluation as a 
purpose for its creation. Broadly defined, evaluations are objective assessments 
of program and management performance. Specific approaches adopted, c^sts 
incurred, and study objectives v^ry, but all ED evaluation activities share the 
common purpose of trying to ensure that Federal education monies are being 
efficiently spent on programs that are successfully meeting their congressional 
mandates. 



U.S. OFFIC'E OF EDUCATION 

Education evaluation requirements were first mandated by the Congress in 1965 
under Title I of the Elementary and Secondary Education Act. This legislation 
included a provision that school district grantees evaluate the effectiveness of 
their prpgrams. Other OE programs soon contained evaluation requirements. as 
well. The FY 1970 budget was the first to contain a specific line-item for 
Evaluation funds. That year OE also centralized evaluation activities in the 
newly established Office of Program Planning and .Evaluation, later renamed the 
Office of Evaluation and Dissemination. This office assumed the lead in « 
directing an increased number of national education evaluations- The staff 
designed studies, selected contractors to do fieldwork and analyses, monitored 
contractor performance, and reported on study findings an4 recommendations. 
From 1970-79, OPPE spent an average of $18.^ million a year on evaluation 
activities. * - , 

Not all education evaluation activity took place in the Office of Evaluation and 
Dissemination or its predecessor offices. The Bureau of Education for the 
Handicapped sponsored studies on the implementation of the Education for All 
Handicapped Children Act* the Bureau of Student Financial Assistance conducted 
quality control studies of the Basic Educational Opportunity Grant program; and 
the National Institute of Education and the Follow .Tfirough program supported 
^evaluation research and some program evaluations- t 

U.S. DEPARTMENT OF EDUCATION 

During th^ 1970's, evaluation became a critical component of federally sponsored, 
education programs, and evaluation is retaining its prominent position in the 
Department of Education as the 1980's unfold. ED evaluation activities are the 
responsibility of the Office of Management under th^Deputy Assistant Secretary 
for Evaluation and Program Management. Several reasons prompted this 
organizational response, to the new Department's evaluation mission. 
Organizational separation between the offices in charge of prografn operations 
-and the assessment of those programs were deemed essential to maintain the 

O 



independence and objectivity of program reviews. Second, the placement of 
evaluation activities within the Office of Management reflects a departmental 
commitment to translate evaluation findings into management improvements 
whenever possible. The Office of Evaluation and Program Management (OEPM) 
has undertaken a series of initiatives that bring together evaluation, policy, and 
budget staffs to explore the implications of program evaluations on all aspects of 
current idepart mental activities and to incorporate evaluation results into future 
planning. Finally, a number of management evaluation efforts, als<? highlighted 
ir^this.feport, gather information on program operations that can be used in 
conjunction with the more elaborate program, evaluation studies/ Placing 
management and program evaluation activities within the same office allows the 
Department to draw more effectively upon a variety of evaluation techniques to 
m^et specific information needs. > 

• * 

Because of their complexity and data collection requirements, impact and 
process evaluations (definitions and discussion follow) are conducted by outside 
research organizations. These studies usually assess the national impact of 
programs and/or the manner in which t£iey have bee^implemented. They are 
administered by three divisions in the Qffife of Evaluation and Program # 
Management: the Elementary and Secondary Programs Division; the 
Postsecondary Programs Division; and the Occupational, Handicapped, and 
Developmental Programs Division. * ' 

The functions of these program evaluation divisions include: 

o Reviewing the state of knowledge about pVograms and identifying 
program information needs. 

o Preparing study designs and contractor selection criteria. 

0 

o Directing the progress of contracted work with particular 

^ttentiorkto the implementation of the original study design and 
technical direction to guide changes in design (this involves the 
selection cfnd development of measuring instruments, sampling, data 
collection, and analysis). 

o , Preparing appropriate executive summaries to communicate findings 
« to ED policymakers, the Congress, and the public. 

. 

These three divisions constitute tfte Office of Program EvAluation r o ne of tw a — ^_ 
major offices reporting to the Deputy Assistant Secretary for Evaluation and 
Program Management. ' fc ' 

The counterpart office under the^same Deputy is the Office of Management 
Evaluation* Several critical Department management functions are also housed 
within this office which includes the Divisions of Program Assessment, Quality 
Assurance, Organizational Development, Management Analysis, and Education 
Data Control. Among their responsibilities are:* 

r 

o management analyses • 

— o quality assurance studies ~ * 

o needs analyses for office automation 

4 0 • V " 
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o work measurement reporting 



o short-term management assessments oi how' well programs * 
are being, delivered to recipients 

o organizational development planning 

~~ ed evaDj^tion activities ' 

^ * s 

The concept of evaluation has been broadened considerably in the Department of,. 
Education* New techniques and approaches h^ve been developed to increase the 
capabilities of Department policymakers and program officials to meet program 
information needs under budget and time constraints. Before explaining the new 
techniques or the old, one must review how the Department goes about 
establishing an evaluation agenda. The evaluation planning process structures 
the agenda and largely determines the choice of evaluation techniques to be 
used. — 

Evaluation Planning 

Evaluation information is necessary in ordei/to answer one or more critical 
question? about education programs. These questions je main at the heart of the 
evaluation planning process: 

o j-— Who is served by the program and' who needs service? 

o What are the services, how well are they deliyered, and * 
w&at do they cost? 

o " What are the effects of services on recipients? 

o* ' -What are the costs and benefits of alternatives? 

o What improvements Should be mAde in the management and 

administration of th? programs? 

* v. » 

o What program practices, projects, or products ^nsjtffective 
, in achieving program objectives?. 

Every education program would profit greatly with answers to questions* 
Budgetary constraints, however, have meant that programs cotjid only be 
evaluated on a selective basis* Because of "competing program demands, 
evaluation planning began in the Office-of Education* 

In OE, the strategy originally use^ to identify program evaluation needs was for 
the central evaluation staff ia the* Office of Program Evaluation to request 
anhually formal recommendations for evaluation projects from program 
managers and then develdjp a recommended list of projects to be undertaken in 
the following year. Th'is Hit was submitted to the Assistant Secretary for 
Planning and Evaluation (ASPE) in the former Department of Health, Education, 
.and Welfjare (HEW) for approval; , * 




In January 1979, OE initiated a new procedure to develop the annual evaluation 
plan* The purpose of the new procedure was ta exert more contr<jI over 
evaluation planning and broaden the review process* An Evaluation Planning 
Committee was established consisting of several OE and HEW policy off icials. 
This group prepared a comprehensive plan for ASPE's approval* The expansion of 
OE's evaluation planning and review process was generally regarded as a useful 
development in that it helped focus OE planning on such critical factors as 
rogram reauthorization ancf specific congressional or departmental concerns. 

O^Te problem that any evaluation group must solve is how to ensure the 
timeliness of their evaluation findings for decisionmakers* A common criticism 
of most Federal evaluation offices has been that their study findings lose 
relevance to Administration and congressional policy officials because results 
are reported too late to be fully used in reauthorizing programs or deciding on 
budget requests or appropriation levels* In response to this problem, the Office 
of Evaluation and Program Management has developed an evaluation planning 
model* Its primary purpose is to help ED focus on the type and timing of 
program' and policy Questions to be addressed by evaluation staff /so as to meet 
the information needs of executive branch and congressional policy officials. 
The approach adopted for the model is substantially more systematic and 
comprehensive than previous efforts* In addition to helping ensure that 
appropriate information from* management and program evaluations is available, 
the planning model can also be used for planning, research, and statistical studies 
conducted by other ED offices. • ■ 

The context for the evaluation planning model is the ,r hf e cycle/' as authorize^ 
in legislation, of individual Federal education programs or groups of programs 1 
ur\der consideration. Life cycles begin approximately, 2 years before a given 
program is to be reauthorized and continue ntil the following reauthorization. 
Accordingly, a typical life cycle is 5 to 7 years, depending upon the length of 
reauthorization. ~ 



There are two reasons why the life cycle add, consequently, activities under the 
evaluation planning'raodel start even earlier than the beginning of a 
reauthorization penod* One is the importance of specifying in advance the 
information needs pertaining to futurp legislation, management, and budge\ 
issues and identifying the sources for each type of information. This 
specification must be done far enough in advance for each information-producing 
activity to J>e planned, carried out, analyzed, and reported in a timely fashion. 
The second reason is to foster early coordination o^information-producing 
activities^ thereby minimizing gaps or duplication of effort and to ensure that 
requests to the field for data are not excessive. The evaluation planning model 
also recognizes the need for flexibility during the program's life cycle as the 
information agenda changes and corresponding shifts ars made in the evaluation 
plan, so that management and program reports will be timely, relevant, and of 
the greatest possible use. In addition, there are a vahety of uses for evaluation 
findings asiderfrom reauthorization efforts. For example, Department managers 
use evaluations to improve grant-making and other processing operations while 
the budget process demands information ffom evaluations for Department, OMB, 
and congressional officials. As information needs change*during a program's life 
cycle, revisions in the application of the evaluation planning model to particular 
program areas Can be made. Overall, the planning model serves as the guide for 
selecting among various evaluation techniques to fulfill the information agenda. 




Traditional Evaluation Activities 



Evaluation information on national programs is essential for responsible 
management. Without it, program effectiveness would remain unknown or 
aneadqtal and improvements in service delivery would be hampered. A variety 
9f information-gathering techniques or approaches are required in order to meet 
specific management needs. Some can be categorized, as traditional ir^-tha* they 
have t>een developed by evaluators over a relatively long period (15-20 years) and 
are viewed by many professionals as the "standard" way of gathering program 
information* These activities are called process or impact evaluations* y 

Process evaluations include studies of the*activities, operations, organization, 
and other aspects of a program* They may be designed to examine reported 
shortcomings in program operations or merely to, monitor program 
implementation Less elaborate varieties may involye short site visits and 
management case studies. The more/elaborate approaches can involve careful 
^ measurement of how many individuals receive services or how institutions 
comply with Federal mandates The main focus of such activity is understanding 
the delivery of program services and adherence to standards, rather than 
understanding or estimating effects of service. t „ 

Impact evaluations of education programs attempt to estimate as accurately as 
possible the direct effects of a program on its main target group, whether 
children, adults, pr institutions. These studies are more difficult when there are 
multiple target groups and when many program effects afe indirect.. If costs and 
effects of program variations are estimated, the activity may involve cost- 
benefit analyses* . * ; 

* 

During FY 1980, 13 impact evaluations were in progress. The studies were 
Conducted by highly qualified, professional research ^irn^s or universities 
according to explicit requirements detailecfby the.Departmeot's evaluation staff. 
It may be asked why Federal employees do not Carry out these projects rather 
than work through an outside Contractor. The/answer is that program 
information needs (both in terms of the number of programs that are evaluated 
and the complexity of each evaluation) and the consequent collection of data in 
the field far outstrip the relatively limited evaluation staff resources within the 
Department. * * 

Although they do not actually lead a research team in the field, departmental 
evaluation staff provide technical direction and control from the beginning to 
the conclusion of a process or impact evaluation. Initially, a staff member is 
designated as the project monitor and develops the specificatibns for the work to 
be performed by a contractor. Once the contractor is selected through a 
competitive bid process and the evaluation is initiated, the project monitor 
maintains close contact with the contfector throughout each phase of the study. 
The basic purposes for this monitoring function are: (1) to make sure that the 
study ^is conducted in accordance with the original design or approved changes; 
(2) to advise the Contractor when problems arise which may jeopardize the 
validity of the stuto; and (3) to ensure that the study, as executed/answers the . 
appropriate questions. At the completion of the contract, the monitor takes 
responsibility for the production of an executive summary of the findings, and 
ensures that the contractor's report 15 reviewed by Department officials and is 
Ultimately transmitted to the appropriate audiences. 



The greatest strength of impact and process Evaluations is that they provide the 
.most comprehensive view of program .operations or t accomplishments that tne 
Department can obtain.' On the other hdf\d f two .drawbacks may make them 
inappropriate in certain situations. Because of the elaborate nature of these 
p sfudies, they axe more expensive than other option^ and tost is an even more 
critical iactor today when evaluation budgets have generally been redoced. 
Time is another consideration th^t cafij&ork against impact and process 
evaluations. The^ rpay tak^ several tf y*6rs to complete, and information needs 
are often more immediate because many programed budgetary decisions cannot' 
be^dMerred. m * - ' ✓ 

\ ' ' • * • 

These shortcomings have led evalu^ion officials in the Department to search for. 

othej means to obtain useful evaluatipn information f more quickly and at less 

cosV Thetesult Is a series of ,evaluetion/r(}anagement innovations that 

supplement impact and process evaluation capabilities within OEPM. 

New Evaluation and Management Activities * \ * 

Additional program information is being obtained thr&jgh four evaluation 
techniques new to the Department pf Education. Evaluability assessments and 
the program objectives, yjfctem focus on developing a broad consensus regarding 
iong-range, measurable program goals ^nd otJjectiv&s/ P^okram management 
reviews and secvicg delivery assessments are low-cost evaluation tools which 
Department staff als<jljse tq bbtain program information. In addition, 
management*evaIuation techniques are use^ to review and analyze the 
effectiveness and efficiency o£#ED's management practices, systems, 
organisation, and procedures, anfUtt) prescribe improvements where appropriate, 
these techniques include: manafcem&ft analyses ; quality assurance studies; and 
organisational de^lopm^nt activiti^? ] 

l.^JEval^ability Assessment (EA) 

Evaiuabiht^^assessment is a technique designed to support the evaluartion process 
by first ensunn&4iat a program has a solid management fouQ<jation%EA's 
purposes are to*fletermme the extent to which a program is ready for evaluation? 
what changes might be^jieeded to make the program more manageable and 
accountabfe; and how an evaluation of the orogram migj}t be most usefully w * 
conducted* ft fully successful evaluability assessn?er\t results in: (1) clearly 
specified and agreed upoxvprogram objectives and activities; (2) an explicit 
statement of fhe assumptions that underlie tfie-prggramj (3) a list of program 
performance indicators that Sire agreed upoji by those responsible for the 
program; and (*) a detailed plan for * v 

o What are management's program objectives and expectations? 
What resources, activities, objectives, and assumptions make up 
management's intended program? 

o What, in the view of policymakers (executive and legislative^ 
is the program e'xpected to accomplish,' what are acceptable 
j indicators of performance, and how will program inforrpa- 
^ * tion be applied to them? * , 

<x Wha^are the likely uses of ^iformaticjh on program performance 
* ) at each management level? What range of actions might 
management <X>*nsider*a5 a result of yarious findings? 

V •• ' v . " ' . -»v ..is-'. • J*' .. 
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On the basis of responses to these questions and a review of relevant documents 
f (e.g.y the authorising legislation and legislative history), the assessment team 
develops three types of descriptive charts or models of the program. A logic 
model is drafted to represent .the intended logic of tjie program (e.g., if eventA 
occurs, then it is assumed that evervt B will occur). A second set of models, " 
known as function models,' trace the program's processes, including suc;h events 
as ftows of activities, people, money, and information/ The third modej deals 
with measurement of progress toward program objectives* It identifies measures 
which could be taken at various points in th& process. to indicate program 
performance. Since ail these models arje based on interviews withJE&defal 
officials only, they represent a description of the intended program . 

The assessment team then visits a limited sample of projects i to obtain 
information about the actual program as it operates in the held. A second set of 
models is developed to represent program reality. Questions addressed during 
and after the field visits include: £ 

o What are the program inputs^activijies, and outcomes 

based 'on a review of actual operations at a sample of project 
+ sites? t v , 

o What measurements and comparisons are feasible, given existing 
data systems? « * s~ ' 

o * What data are obtainable on program performance? Are there 

data sources for management's *agreed-updn .measures of progress? 

All information gathering and modeling at the poltgy, management, and field 

operations levels Jay the groundwork for the analytical stage of the assessment. 

Questions asked by, the team during the analysis of program evaJuability includes 

■ * 

' & Is program jnanagemenj's description wejtl-defined? V ' 

• * 4 * 
o ^ Is maMHte^nt's descriptiQnacceptable to policymakers? 




o - ^ Is manJglWent's description a sound representation of the 
pft>gram in the field? B • 

i 

o Are management's expectations plausible — that is^ do they 
*\ appear likely to be accomplished? t 

o . What portion of the intended program is ready for useful 
evaluation? • - 1 



°* \ What management options can be suggested to improve program 
^performance? 




r . B rma 

performance and measuring accomplishments. 

Like full-scale impa.ct and process evaluations, EA's are conducted by an outside 
t£am of-researchers subject 'to close Supervision by a Department, project 



monitor* Evaiuab^Itty assessments differ significantly in that they are intended 
to produce resglts relatively quickly (6 months) andmexpensively{an average 
cost of $65,000)- Their greatest benefit, howeyer, is the savings of time and 
money that might otherwise be expended initially on an unsuccessful impact or 
process evaluation « that is, the resources that might be used to evaluate a 
program that instead is shown by an EA to lack agreed-upon, measurable 
objectives* 

Twelve EA studies of ED programs were initiated,in FY 1979*and FY 198Cl*At of 
the end of FY 1980, work was still in process on two of the assessments (the 
Strengthening Developing Institutions program and the Women's Educational « 
^Equity program)* two others had been terminated before completion (the 
Vocational Education program and National Center for Education St^tistics)^ 
eight of*the assessments had been completed. These eight studies covered the 
following programs: Bilingual Educationf Career Education; Cooperative 
Education; Early Childhood Education for the Handicapped; Follow Through; 
Institute for Museunn Studies (General Operating Support); Language Training and 
^rea Studies; and Vocational Rehabilitation* At least four new assessments will 
be undertaken in FY 4981. • 

V 

2* The -Program Objectives System 

The Office of Evaluation and Program Management 15 currently developing a 
system that is less intensive, ttme^onsurning, and expensive than evaluabihty 
assessment but, like EA's, is intended to generate measurable program objectives 
an& indicators 61 achievement* The process will place primary responsiblity on 
program managers for: (1) identifying key objectives; (2) developing measures or 
^dicatQj^^hidxj^aoj^ 

departmental and congressional review* OEPM will be responsible for; installing 
and managing the system; defining terms and setting criteria for objectives; 
providing technical assistance and necessary support to program p managers; 
coordinating review of objectives and indicators; and advising on the general 
quality and appropriateness of final products* 

It is hoped that this program objective-setting effort will produce objectives and 
indicators for programs just prior to ED's internal budgeting cycle so that the 
objectives are useful in the budget review and policy activities of the 
Department* In.addttion, the Department wUi be able to imprpve significantly 
Us efforts to meet the mandate of Section k\7 of the General Education 
Provisions Act requiring development of specific; objectives and performance 
indicators for all £D programs. 

The objectives and indjcat6rs developed through this process wilLbe transmitted 
to the Congress in the Annual Evaluation Report , It is anticipated that the 
transmission of these individual program objectives and measures will assist the 
Congress and the Department in agreeing upon legislative and administrative 
mandates for program operations* Some differences will naturally emerge in this 
dialogue, but the existence the initial objectives should be helpful in clarifying 
and resolving those differences* # , > 

: * * ' 

Programs which are especially complex or controversial will continue to have 
objectives developed through the more- analytical, intensive process of 
evaluabihty assessment* The program objectives system should be extremely 
cost-efficient because, once developed, the entire process can be operated 

• 10 ' lb 



exclusively byjDepartment management and program staff. Through the use of 
both methods, w the Dep&ttment hopes, to improve the management of its services 
and programs. , 

t 

3. Program \fanagement^?views{PMR> , 

Ahis evaluative technique is also being developed'apd refined by the Office of 
Evaluation and Program Management. It is a study designed to synthesize 
. ^ existing program information in a short period of time and to supplement that 
information with independent, short-term assessments covering important 
knowledge gaps aboJ)T the operations of the program. The resulting report gives 
the Secretary and ojfiefr top managers a* complete but succinct picture 'of a 
program including, i\ s history, legislatiye goals and objectives, resources, J 
administrative and»njanagement operations, field perceptions, effectiveness, 
: issues, and problems/ * ^ **** ' } 

Program management'revitws will be particularly valuable: when programs are 
facing reauthorization without Sufficient data on eertain aspects of their 
, operations^ when controversial issues require difficult management decisions* 
and svhen top management changes are made. While PMR's are not designed t<5 
develop ^uick solutionsior program problems, they can be used as diagnostic 
tools to identify wtiere administrative, policy, or legislative changes should be 

* made. % 

* * • ■ • 
r f A PMR will 6e "directed by OEPM staff with the; assistance of other program and 
staff offices within ED. Staff with expertise in management and program 
analysis, work me&uremsftWjrganizationaj behavior, qualify control techniques, 
and othtftfieJds make up a PMR team. * 

The specific purpoj*, scope, lewd' of detail, and length of each PMR will be 
defined by theOffjce of Evaluatiqfi^d Program Management in conjunction 
with top management for the program. Because each PMR* will require a 
significant level of resources, no more than three or four such studies will be 
undertaken in a fiscal year. Use of the PMR technique will be reserved for high 
priority, needs and interests of the Secretary and Assistant Secretaries* 

\ Service Delivery Assessments <SDA) 

* 

# . These short-term r current assessments of ED programs and program-related 
issues are conducted for the immediate use of the Secretary and Under 
Secretary. TKe t 3* fb 5- month studies collect opinions and suggestions* from 
persons who directly receive or provide-swviees-under ED programs in order to 
gain an Understanding of how programs are perceived across the country. An 
SDA gauges; how successful EP programs are perceived to be at the State and 
local levels in meeting the needs of students, teachers, parents, and 
administrators; what problems exist as programs are being implemented^ and 

* what improvements are identified by those directly affected by the Federal 
•programs. ' * 

* * 

It is important to understand that SDA f s are limited activities that do not 
.correspond to t/adltional evaluajion^tudies, audits, compliance reviews, or 
monitoring activities. While they often employ similar methods, SDA's are more 
analogous to in-depth analytical reporting which uses open-ended discussions 
q th people in local settings. The knowledge gathered is generally subjective and 
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qualitative in nature* BeCause of these characteristics, SDA findings are not 
intended for. general distribution but rather foriuse as one of many program 
information sources available to the Secretary^ncT other decisionmakers* 



The reasons for undertaking an individual *>DA vary with each study* Some 
frequent reasons include; suspected operational problems* significant changes 
planned or underway; expiring or proposed legislation; plans for a major 
initiative; programs or issues the Secretary wants to explore. The SDA tiams 
seek especially to identify operational improvements which the Secretary and 
program officials^can make without the need for legislative, regulatory, or 
budgetacy change* In this fashion, the SDA findings and recomnfphdations can 
v be implemented immediately. 

In the Departmen^t>f Education, SDA's w^ill be conducted under the overall 
supervision of OEPM with regional staff providing onsite support necessary to 
conduct the studies. It is hoped that State and loc^Jspducation staff will 
eventually participate with ED regional staff in planning and executing these 
service delivery assessments. ^ 

y Management Analyses ^ 

Management analyses are studies intended to improve quality control and 
productivity in Department management and program operations.. Improving 
overall productivity is viewed as a- crucial responsibility for tQpMeW career 
management in the Department. Recently, management analyses Have led to 
recommendations intended to increase efficiency and reduce costs 10 three 

areas: travel; personnel actions; and student financial aid. \ 

* i 

The emphasis m ail three areas is on ^making accountability ^nd quality control 
integral parts <$f these^rocessing and delivery systems. Scholarship aid, travel 
documents, or personnel papers do not result from a series of discrete ^ 
managenai tasks but rather are produced by an interdependent system where 
efficiency or mefficiertcy at any one point affetts the entire process. This 
comprehensive perspective fosters accountability by allowing every individual to 
share responsibility for the efficient delivery of services. I 

v I & ! 

Management analysis also entails the study of the impact arfid applicability of 

technology, particulariy,in an office setting. Research andjdevelopment efforts 

in office automation attempt to avoid the common pitfall of assuming that new 

equipment by itself will increase productivity and efficiency* New .equipment 

may merely hide existing administrative problems. Therefore, managtfnent 

analysts first examine existing office operations and endeavor_to make these a^ 

productive as fjpssible. The analysts may subsequently recommend that new 

equipment be added, but only based on sound productivity f pais and a thorough 

understanding of existing office Capabilities. 

6. Quality Assurance Studies 

Quality assurance studies are intended to assess the accuracy <and reliability of 
program data and information used for management decisions, ^lultidisciplinary 
teams composed of statisticians, economists, systems analysts, management g 
analysts, and attorneys use a vanity of evaluation and stalistical techniques \to 
assess departmental support systems ai\d programs. Three 
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quality assurance 
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studies currently in progress focus on: grantee monitoring; the process of , 
evaluating discretionary grant applications; and developing consistent program 
descriptions* „ % 

Previous studies conducted by the Office of Education revealed the absence of 
an agency policy with respect to grantee monitoring. These studies- indicated * 
that there were no minimum standards underlying monitoring efforts nor any 
mechanism in place to encourage consistency' in monitoring. These studies 
contained a variety of recommendations to resolve these problems.,, The Office 
of Evaluation and Program Management's Quality Assurance Division is currently 
using these recommendations as the basis for the development of directives, 
manuals, and other pdlicy documents related to the monitoring of State formula 
and discretionary grant programs in ED. 

s 

A quality assurance survey is also underway of all discretionary grant programs 
in the Department (approximately 80) ^hat use nongovernment expert! to review 
grant applications. The survey and subsequent data analyses are intended to 
gather baseline information on existing procedures and then suggest ways for 
improving the system's shortcomings. The survey explores topics such as; 1) the 
review process stage at which outside/inside reviewers are used; 2) the use and 
consistency of evaluation standards ihat reviewers follow; 3) the legislative 
requirements for using grant application reviewers; artd the alternatives to 
using nongovernment reviewers. t 

The Office of Education did not have an automated systerp for providing 
consistent and reliable responses to frequently asked questions about education 
programs. To remedy this situation, its quail ^assurance Staff is developing a 
Program-Profile System (PPS), a computerized information retaeval system 
containing common key descriptors on each Department program. PPS is being 
designed to generate indices of programs by descriptors as well as to generate a 
listing or profile of all descriptors for each program. PPS will be developed in 
^ phases; Phase I will profile programs by the Catalogue of Federal Domestic 
Assistance number on 18 descriptors; Phase II will profile subgrant-making 
activities and contract programs; and Phase III will add new descriptors for each 
program. * • * 

• 

7. Organizational Development Activities • • * 

Organizational development activities include the creation of an administrative 
communications system, and work measurement and organizational effectiveness 
studies. The administratis communications system is departmentwide. It is 

. used for coordinating the development and distribution of lntqnj^l management 
directives, Secretary-level circulars, administrative announcements, and the ED 
organization manual. *The pur{>oses of the system are to; 1) assure that affected 
parties have ample opportunity to participate in formulating pernjapent 
administrative policies and procedures; 2) provide complete, concise, current, 
and authoritative information to agency personnel* 3) produce easy-to-read and 

* understandable documents; 4) jjroup in a single source for each specialized 
audience information on specific subjects) and 5) make information easy to 
locate/ * : n 

Work measurement studies collect current data for a departmental workload 
reporting system that statistically compares personnel resource needs on the 
O sis of m^jor output indicators. Within each general category of programs 
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(State plan, formula, or discretionary programs), the following npajor indicators 
are computed for each program: number of applications received; number of 
grants made; number of dollars appropriated; and number of onsite reviews 
conducted each year. The system will assist top management in assessing 
workload trends and identifying potential areas for redistribution of positions. 
Other work measurement studies can be integrated with management 
improvement efforts in such areas as objective-setting, quality assurance, and 
system design. * " • 

Organizational effectiveness studies assist individuals and ED units in working 
more effectively and productively* Projects are designed to examine such issues 
as diagnosing organizational problems, clarifying communications, establishing 
goals, working with conflict, and improving meetings, decisionmaking, and 
-^interpersonal relations. - 

The services v of the organizational development staff include: ^leadership 
transition assistance to aid new leaders and key -management staff to take 
command quickly and work effectively with the bureaucracy; 2) organizational 
diagnosis studies and feedback sessions; 3) consulting on agency trouble spots; 
and 4) designing and conducting team-building activities for leaders, managers, ■ 
and they- staffs. 



Developing a Systematic Approach to Evaluation 

Having developed an evaluation planning model, continued the conduct of 
traditional process and impact evaluations, and instituted new, shorter term 
evaluation technique, the Office of Evaluation 'and Program Management has 
• reshaped the evaluation function and operation in the Department. The 
evaluation planning model helps predict information needs it given pointsfdgring 
k program's life cycle. A wider range of evaluation techniques can now be 
systematically Selected with specific program data and information requirements 
in mind. Finally, this approach can lend itself to better plartning for information 
collection activitifl»other than evaluation studies; e.g., research, statistical 



surveys, and planm^ studies* . 

As an example of how these evaluation techniques and the planning model can 
assist ED officials in obtaining tirpely and useful evaluation findings, take a 
hy poetical, newly authorized postsecondafy student assistance program which 
might require the following types of information during its life cycle. 

^ „ o ' clarification of program objectives and performance measures 
through either the program'objective-setting process or an 
evaluability assessment; timing : planned shortly before final 
■ . authorization and executed immediately after the law is enacted. 

/ 

o design of a delivery system for student assistance using manage- 
ment analysis, ofganiz&tionaf development, and quality control 
expertise; original inclusion of a quality control 
system giving feedback information to managers on how well the 
program is working is especially critical at this point; timing: 
planned before final authorization and executed* shoftly after 
enactment of the law; the delivery system is dependent ^ 
upon perceived program objectives and should incorporate perfor- 
q * mance measures as part of its'management information system. 
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o special fnforntatlon will be needed for annual budget reviews or 
* upob request of Department officials; to meet those demands, 
% Shorter term evaluation tools such as service dellvery'assess- ^ 

ments, program management reviews, organizational studies, work 
measurement reviews, and management. analyses can be planned 
during the^ program's life cycle; timing ; usually after the program has 
.been in operation for*l to 2 years^ 

A o major issues surrounding a program may require larger evaluation 
9 projetts, such as those dealing with effects of Federal funds on 
State and local education agencies, assessment of the national 
strategy used for implementating a specific program, or 
the national impact of a program on t(ie educational achievement of 
. K children; timing ; depends upon when the information is needed ~ 

for example, a national impact study whose'results are needed for the 
t . next reauthorization cycle^hould report te/findings approximately 

^ 2 ypars before reauthorization so that Department officials will have 

the findings as they draft proposed legislation. This, in turn, means 
that the study should begin 3 to»5 years brfore the reporting ^ 
date so as to have enoi/gh^ime to colfect anifanalyze the data. 

* • 

# * # 

An illustration of a typical evaluation planning model that mrght be developed 
for such a newly authorized program is^hown on thfefollowing page. 

Dissemination of Evaluation Findings ^ 

A diverse audience awaits the results of BDpvaluation activities. *The most 
directly concerned are policymakers and program managers Within the A 
Department and the congressional committees which have jurisdiction over / 
education programs (principally, the House and Senate Appropriations / 
Committees, the House Education apd Labor Committee, and the Senate Labor 
and Human Resources Committee), Local and State education officials, 
education-related public interest groups, parents, and a sizable community of 
evaluators show varying degrees of interest depending upon the evaluation in 
, question. Communicating evaluation fadings in ji. timely and useful fashion to 
responsible officios and an interested education community is, critical to the 
success of the overall evaluation effort. 

ED has adopted several strategies tp ensure that Its evaluation f esults are 
disseminated rapidly to influence subsequent program decisions or inform the 
education commuhity. One new approach for reaching the broader education 
community with useful evaluation reports is this general circulation volume * 
which, used In conjunction with volume H, provides a comphrehensive overview 
of ED programs, the evaluation process, important new findings, and evaluation 
uses. - „ 

Several alternatives have be£n developed to assist the dissemination process, 
within the Department. A new internal review procedure guarantees all key 
Department officials an opportunity to review and comment on the results of an 
evaluation study witnin strict time limits. Thf entire review is completed within 
a short period of time; comments or objections from. reviewing Officials may be 
incorporated into the final executive summary of the study. * 
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EVALUATION PLANNING MODEL 
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Regular st«£ briefings on evaluat*oaiindings can also be an effective 
mechanism for communicating evaluation information* within the Department. 
For example, OEPM of fefs to brief each program office on. the budget 
implications of evaluation findings. The budget briefing is keyed to the 
beginning of the annual budget-preparation cycle. The evaluation staff also 
nieet with program officialsjo discuss the implications of evaluation findings < 
appropriations and legislative reauthorization hearings before congressional 
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A newly established Program Resource Center within OEPM represents anothet 
way irfwhich information on programs administered.by ED wil^be made available 
to members of the Department. The center will collpet and index all 
managements and program evaluations conducted by ED or other Federal agencies 
(e.g., the General Accounting Office, the Congressional Budget Office) that 
perta'in to Department programs or operations. Other program documents will 
be added as staff resources and time permit. At a minimum, the resource center 
will maintain copies of all final evaluation reports and executive summaries and 
other key evaluation documents referenced in the current edition qf the Annual 
Evaluation Report . Eventually program files coutfi include legislation, Rlanning 
studies, polity analyses, program regulations, program guidelines, and annual 
program reports to the President, th* Congress, br the Secretary of ED. Plans 
are underway to have as many documents as possible recorded on microfiche or 
magnetic tape for convenient access and long-term storage. 

Efficient us£ of evaluation findings has become a top management priority in 
ED. Communicating findings quickly is an effective, indirect way of achieving 
this objective. A % more direct approach for ^valuation folio wup has also been 
developed on an experimental basis. When an evaluation report is likely to have 
a bearing on program legislation, budget decisions, major program operations, or 
management practices, the Office of Evaluation and Program Management, in . 
cooperation with the Assistant Secretary for Planning and Budget and the 
appropriate program's Assistant Secretary, will establish An Evaluation 
Implementation Board- The Board, consisting of senior evaluation, planning and 
budget, legislation, legal, and program staff, will review the evaluation report 
and synthesize its findings into a decision memo for the Secretary, Under 
Secretary, or appropriate Assistant Secretary. The memo will contain policy- 

* relevant and management improvement options and recommendations. Once 
decisions have been made, the Board will also develop an implemeqtation time 

^schedule. > 



CONCLUSION . 

Since *t he Department's creation, the evaluation function has changed direction. 
The scope of evaluation interest has been broadened to include all aspects of 
program operations. Program impacts are assessed as part of a complex pcocess 
th?rt includes ongoing program operations, budget, and management activity 
Innovative management and (evaluation techniques ^are now in place bea 
traditional evaluative methods; Greater attention* is paid to systematic 
evaluation planning and comprehensive evaluation (oliowup. The overall result is 
a flexible, responsive, and useful evaluation capabili 
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NOTES 



1 These activities are discussed later in this chapter. 



^Two evaluability assessments that were begun in FY- 1979' under the 
Department of Health t Education, and Welfare have been terminated. Managers 
of the Vocational Education Programed the National Center for Education 
Statistics asked that the assessments be stopped because the heavy workloads 
imposed by th6 transition into the Department of Education prevented their 
participation. Because of organizational and staff ing changes which have since 
taken piace t it would be necessary to start new assessments rather tftfn attempt 
to finish either of these studies* 

* « 
_ 3 They include: validation and measurement .techniques; statistical quality 
control charts; regression analysis; and sampling techniques. , 
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This chapter is an overview of ED program activities and evaluation findings that 4 
highlight program operations. The underlying theme of the following materials isf 
diversity — diversity in the typg?of programs ED funds, intended recipients, anil 
the mechanics of program operations. Not only do programs vary widely, but 
occasionally so do implementation procedures within a slngle*program. Nearly 
all venations result fh>m the flexibility in program operations that the Congress 
mandates; States and localities are expressly given latitude in program 
implementation. In addition, the operations of State and local education 
agencies and universities differ enough that even a common legislative goal may 
translate into substantially different procedures arid, perhaps, even results, 

c - . \ 

Understanding the diversity between ED programs, and occasionally within them, 
is one of the principal objectives of the Department's evaluation activities. It is 
also one of the greatest challenges because few definitive answers or 

* generalizations result frorp this work. As will become evident in subsequent 
sections of this chapter, conclusions from even the mQst thoroughgoing and 
comprehensive evaluation studies are often reported \yith caveats. Caution is 
usually in order because of the changes expected to result from the influence of 
Unfunded programs. Overall educational achievement, for example, is 
influenced by a multitude of factors including un^quejfndmdual, family, and 
community characteristics. Many of these factors tannot measured; others 

* cannot be controlled statistically. m **f* «v 

4 

Wfyle caution is clearly called for in interpreting broad educational change as a 
result o^grogram activity, fairly reliable assessmentS'of immediate- program 
impact afeTJo^sible. Subsequent sectipns describe such assessments. The 
elementary and secondary school overview discusses programs for low-achieving 
students, handicapped by poverty or a limited ability to speak Enjjjish. Recent 
evaluation findings indicate that fthilcfederal education funds have reached 
these intended recipients more effectively m recent yearns, there is still room for 
improvement. Progress has also been made in program implementation. 
Program impact on improving the academic skills of elementary and secondary 
school students is seen to vary by subject matter and grade level. j 

The overview of postsecortdary education programs focuses on enhancing 
educational oppprtunity by providing Federal financial support to students and 
institutions. Over time, significant changes and differences are found in 
attendance at institutioi)s~of higher education by students' sex, family income, 
and race. The goals of Federal grant, loan, and work programs are to improve 
student access to and persistence in higher education and provide a range of 
institutional cho^fts. This section assesses the degree to which Federal funding 
has succeeded in achievingtjese goals bj^utilizing the concept of student 
burden— the dif ference^fween the cost of attending a particular college ancf 
Federal aid receivejj/rederal aid to. institutions also af fects.^ludents' ' 
postsecondary school opportunities. These influences along, tffljth student transfer 
patterns amon^ higher education institutions are briefly reviewed m this section. 

The third and final section of chapter 2 is an overview of the large and diverse 
number of education programs directed toward special recipient groups or 
particular educational needs. These include programs to improve the quality of. 
educational practices} expand learning resources; foster occupation*, career, 
and adult education; and assist the handicapped and Native Americans in 
receiving quality education. The results of evaluation studies in this,section are 
wide-ranging as the programs themselves. One general statement is 

s 



applicable however; foods for evaluating thesi special category prd&rams have 
been extremel/limited* A feVhave special evaluation monies (set-aside$) 
authorized and appropriated by the Congress . Most do not. Given the relatively 
modest^approprUtion for m?ny of the^e programs and the Departments , 
Extremely limired discretionary funds f many programs have not been formally 
evaluated for these reasons, special category programs will benefit 
significantly ffom the new management improvements initiated in the 
Department and discussed in chapter I. 



ELEMENTARY AND SECONDARY EDUCATION PROGRAMS 

Approximately 4S million students are currently enrolled in elementary and 
secondary schools. About 90 percent attend the 87,000 schools operated by 
nearly 16,000 public schoof Systems; tfie other 10 percent attend non-public 
schools. * % 

Many educators and the general public have expressed concerns about the 
conditions in our public school systems. According to an annual Gallup Poll, the 
public's rating of schools declinectehrougjuthe latter half of the seventies, but 
this decline may have come to a halt in 1980. Although only 28j>erceM of the 
public has a "great deal" of confidence in the public scho^kraccoftfing to tftat 
poll, th%^tandlng of public schools exceeds that of all^p^er important American 
institutfeftrexcept churches. Oi! 

The five major problems confronting public schools, as perceived by thepoU's 
rtSpontfJfcUL*, are: (1) lack of discipline; (2) use of Illegal drugs-, (3) poor curricula 
and standards; (4>iack of proper financial support/; and (5) integration or b islng. 
Lack of discipline was rated as a problem almost twice as often as any other 
topic* » 



/. 



Other public opinion polls register widespread concerns about education/ But for 
many of the problem areas qted, it is difficult to describe the exact nature ot 
the problem, let alone express the degree to which fhe problem is real. Measured 
ment is relatively good, however, for educational achievemenras reported by the 
U.S. Department of Education's National Assessment of Educational Progress 
(NAEP). Every few^years NAEP assesses the educational achievement of 9-year 
, olds, 13-year olds* jand 17-year-olds in major academic subjects. 

\ ' ' is 

Most subjects were mea§ured twice during the I970's and^ at the risk of some 
oversimplification, can be summarized for reading, math, writing mechanics, 
writing quality, science, social studies (9-year olds) and politics (teenagers) as 
follows: ' s 



o 9-year olds (mostly fourth graders) showed slight Improvements in 
ruling, sligh^feclines in math and science and not much change 
^ ¥ €lsewhere. * • ^ 

I 

o I3*year olds (3th graders) and 1 7-year olds (llth graders) showed 
' varying degrees of declines jn knowledge of math, science* and 

politics, and showed essentially no change in reading and writing 
mechanics. ^ 

»* * • 

Part of the public's concern about education may reflect the measured declines 
la performance by teenagers. 2 



Major Federal education programs are aimed at just a/ew of the national 
concerns about elementary and secondary education. The issues to be discussed 
in, this section are education for low achievers (especiaJjB^vhen achievement is 
related to low family income or limited English prof idffcy), school 
desegregation, and general improvement in educational practices. The primary, 
questions are: (1) how well are Federal education funds getting to the intended 
beneficiaries; (2) how well have the programs been implemented; and {3) how 
effecttvely^ave the programs accomplished their objectives? 

How Welb\cg Federal Educati3R Funds Getting to the Intended Beneficiaries'? 

. * 

The efficiency with which Federal funds are targeted is indicated by two 
measures: the proportion of ineligible program participants and the proportion 
of eligibles not served. The latter measure is complicated by the fact that ^he 
percent of eligibles served is determined not just by the efficiency of the 
targeting process but also by the total funds available. If funds are insufficient, 
it is impossible to serve all the eligible population. 

For Title I of the Elementary and Secondary Education Act (ESEA), extensive 
djrta p&taining to targeting ar(e available for children in gra^s 1-6, graces where 
local school districts" spend a high proportion of Title I funds. Results from the 
1976-77 school year show that among students scoring below tRe median on 
* achievement tests, 25 percent (approximately 2.5 million students) participated 
in Title I. Of children scoring above the median, only kYi percent participated in 
Title I. These results indicate that Tittel is fairly well targeted on the low- 
scoring students but that many low achievers (7.3 million students) below the 
median do not participate. 1 

Although the language of Title I seems to stress meeting the needs of low 
achievers, some hold that theJunds should also be focused on children from Jdw- 
income families. Moreover, a high correlation is often assurried between/iow 
achievement and poverty. The data indicate, however, that the correlation (r) is 
jnodes^, ranging from .2 in the first grade to .3 in grades.three to fouf^ . As a 
consequence, even thoygh Title I funds areiairly well-targeted on low achievers, 
about 58 percent of* the participants are.not from poor families (based upon the 
Federat Government's poverty indicator). 

The Title! migrant subprogram offers special services fbr the children of 
migrant workers. A Migrant Student Record Transfer System JMSRTS), usmg^ 
data supplied by school systems, estimates thejiumber of program participants in 
1979 at 522,000. The size of the eligible population is unknown, but it is beHeved 
that not all are served. There are probably two tategories of unserved children; 
first, spme unknown number of migrant chifSren are seldom, if ever, enrolled in 
" schpolj second, from 15 to^ 20 percent qf eligible children enrolled in school are 
not registered on the MSRf S.* <An unknown percent of such children are 
probably not served.) On the othfr side of the targeting question, probably f ewA 
than 2 percent of participating children are ineligible.— 

<!> ■ * 

Some 51,U00 Title I-eligible children are instate institutions for neglected or 
delinquent children, but 1976 data indicated Jhat only 27,000 (53 percent) were 
receiving services. No estimate is available for the number of participating 
ineligibles. In both the migrant and neglected/delinquent programs, the 
differences between the numbers of children served and eligible can largely be 
^♦tribiiled to targeting inefficiencies {not l?ck of funds) sinc?e programs are 'fullv 

UC lde< ?- * ■ » . ' „ . 
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In contrast to Title I, the ESEA Title IV-C Program (Support and Innovation 

Grants fdr Improving Local Educational Practice) is not aimed at a particular 
target gtoup. However, the legislation requires that children attending nonpublic 
schools must receive program benefits on an equitable basis with public school %i 
students* Recent evidence indicates that nonpublic students are participating irr 
programs funded by Title IV-C grants on a less than equitable basis* On the 
other hand, the majority of eligible, nonpublic schools, and presumably their 
students, receive Title IV-B (Instructional Materials and SchoorLibrary 
Resources) benefits for learning materials and library resources. 

* ♦ 
ESEA Title VII (Bilingual Education) is primarily f9r children of limited English 
proficiency* Though the program is funded at an amount greater than a 
demonstration program, it is substantially below the level required to serve all 
eligible children. Therefore, the percent of eltgibles served is not a useful 
indicator of targeting efficiency. However, the percent* of English~j>roficiertt 
children participating^ the program is an appropnateindicator. 

A 1978 study of Title VII found that less than one-third of participating students 
were of limited English proficiency. Although there was no legal restriction 
^gainst participation by English-proficient childreh, a proportion of more than 
two-thirds was widely felt to be too hieh, and the Congress amended the law in 
1978 to limit participation #EnfiU#h-profictent children to ^O percent. There 
has been no systematic check ofthe language proficiency of participating 
children since that change in the taw. 

In the Emergency School Aid Act (ESAA), the targeting issue pertains more to 
school districts than to schools or children. The purpose of the program is to 
encourage the voluntary elimination, reduction, or prevention of minor ity group 
isolation £nd to meet special needs incident to tlje elimination of segregation. 
Several studies have found that most funds for the basic grants subprogram went 
to districts that desegregated long ago. In response to this situation, the 
Congress recently amended the law to add recency of desegregation to the 
criteria for judging grant applications. 

r 

How Well Have the Programs Been Implemented? ' 

It is by now widely acknowledged that social intervention programs are often 

poorly implemented. Federal programs in elementary and secondary educatiort 

have not been exempt from this problem. It is also probably true that many 

evaluation studies of education programs haVfe not given sufficiertt attention to 

assessing implementation. 4 Inadequate technology (e.g., lack of instrumentation, 

data collection problems) and high costs .are the main reasonyfor limited 

measurements of the degree of Implementation. 

■ • 

Nevertheless, most evaluation studies do provide some information about 
implementation. ESEA Title I is a good example. The 1965 version of the law 
was not very specific about congressional intent. Initial direction from the 
former Office of Education was similarly loose. It is not surprising, therefore, 
that much early concern about the program focused on possible misuse of funds 
and actions inconsistent with th'e intent of the Congress. Over the year*, ♦ 
however, both the iaw and program administration have become morp rigorous. 

Current evidence indicates that, on the whole, the Title 1 grant program to 
school distficts is reasonably well implemented* Following the apparent intent 
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of the Congress, local projects tend to emphasize instruction in basic cognitive 
skills. Generally, Title I services supplement the usual services provided to 
disadvantaged children, and in amounts large enough to be clearly noticeable as 
an ^educational program distinct from that offered regular students. Much less is 

known about tKe*proficiency of project management and tfie quality of 
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The Congress seerns to have iprtended that the Title I subprogram for 
institutiOTalizedndglected and delinquent children be similar in purpose and 
operation t'o^he\istrictgfc , ant subprogram. A recent study indicateS'that the 
projects do emphasize instructional services and that the Title I funds are indeed 
supplemental to State funds. Otherwise, there seems to be wide variation in how 
institutions implement the program. Actual receipt of services (as measured, for 
- example, by fcours of reading and m.ath instruction) is less, on the average, for 
Title I students than for non-Title 1 students. The difference seems to exist 
because some institutions do not attach much importance to student attendance 
in the special Title I activities. 

.the implementation question is far different for a program such as ESEA's Title 
IV-C. ffere the keynote is State and local flexibility, and the Congress has 
imposed very few constraints. A recent evaluation shows that management of 
tbe program has fostered flexibility. As a result, there is great variation among 
the States with respect to program administration and types of projects. 
Hov/ever, the program did not, in general, lead to consolidated management of 
formerly 'categorical programs as may have been hoped. 

The Sl^tional Diffusion Program (NtSP) takes a different approach toward 
improving School practice. With thi? relatively small program, a number of 
.educational practices have been identified for which there is evidence of 
effectiveness. Evaluative evidence indicates that the National Diffusion 
Program has been successful in causing many school districts to adopt exemplary 
educational practices. Further, there is evidence that districts frequently adapt 
the practices to local circumstances. At this time, it is not possible to say with 
certainty whether good NDP implementation should be defined as high-fidelity 
adoptions with few local modifications, or vice- versa, or something in-between. 
Indeed, the answer may depend upon the kind of educational practice in question. 
Until -there is further evidence or> the effectiveness of adoptions, the question 
will bd not answered. 

The Emergency School Aid AqJ (ESAA), and especially the basic grants 
subprogram, exemplifies that program implementation depends upon the clarity 
of congressional lptpnt as expressed in legislatiop. Results of several studies 
Tshow ^hat a high proportion of the basic grant funds are used for general 
education assistance or compensatory education rather than for desegregation 
assistance, wh ( ich is the program intent. Although not an indication of illegal 
operations, these findings highlight the loose link between the program purpose 
and the actual use of funds. Iji 1978, the Congrfess amended the ESAA legislation 
to Encourage a closer relationship between the program purpose and the 
'activities funded. - « 

How effectively Have Programs Accomplished Their Objectives 7 

Thart Is rarely, if ever, complete agreement among all members of the Congress 
^or £11 administration officialliabout the objectives of Federal education 
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programs, The following results pertain to the most frequently state^d program 
objectives/ I * 

Schools tend to focus ESEA Title I f unds^on reading and math instruction, 
especially in the elementary grades. Recent evidence (students tested in 1976- 
77) shows that for student reading achievement in grades I, 2, «md 3, Title I is 
effective in that Title I students learn to read at a*r*te .faster" :han would be 
expected without Title I but not fast enough to catch up with regular students. 
In grades fc, 5, and 6, Title I does not impart benefits in reading* Math results 
show positive Title I effects in grades I through 6, The details 9f these results 
are rather complicated and the reader is advised to consult specific repqrts cited 
in the Title I section of volume II, 

Among students in Title I projects for neglected or delinquent culdwn (students 
tested 1il1977-78), the reading and math scores were about whatj wpuld be 
expected in the absence of Title I; that is, Title I is not having an impact on 
these students in those skill areas, 

! 

Information about the ESEA Title VII program comes from a study (students 
tested in J975-76-77) of Spanish-English projects in their fourth or fifth year of 
operation. In English Language Arts, achievement gains of Title| VII Hispanic 
stJdents were less than those of non-Title VII Hispanic students,. In math, the 
gains of the two groups were comparable. 

An early (19^73-76) longitudinal evaluation of the ESAA basic grant program found 
evidence of improved academic achievement in el&mentary -schools but no 
improvements the high school level, Empfiasis subsequently shjfted away from 
improved achievement as a program objective and toward improving relations 
among students, A recent study (data collected in 1978-79) found that students 
who received ESAA-funded human relations .services showed greater 
improvement in measures of intergroup relations (e,%,, attitudes, behavior, arid 
self-concept) than students without such services. Evaluations of two other 
ESAA subprograms showed miaed results. One study found that ESAA Magnet 
Schools can be effective when used as a component of a comprehensive, district 
wide desegregation effort and when used in districts with a relatively easy 
desegregation task. Another study showed that organizations receiving grants 
from the ESAA Nonprofit Organization's subprogram were less effective in 
developing community support for desegregation than community groups not 
funded by £SAA, t * 

Federal programs aimed at general improvement in educational practices are 
difficult to evaluate because of the diversity of practices and the sometimes 
intangible changes being sought. Thus, while it is clear that ESEA Title IV-C has 
encouraged State and local education agencies to try alternative educational 
practices, no overall statements can be made about the impact of those changes. 
Likewise, for the National Diffusion Program which seeks change in <a more 
direct and specific way than Title IV-C, broad judgments can nofbe made t 
regarding program impact on students, 
/ " 

Summary 

An important role of program evaluations to inform policymakers about 
program performance in targeting, implementation, and effectiveness. As. 
studies are completed, evidence about each aspect of program performance 
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increases and so does knowledge about how to implement necessary program 
change. It is clear that conclusions about F^dferal education programs in 
elementary and secondary schools are mixed: in.sofne cases (such as district 

* grants of ESEA Title I) funds appear to be reaching the intended beneficiaries, 
providing services as outlined in law and regulations, and leading to, modest 
accomplishment of Overall goals. In others, the funds may be targeted 
appropriately but do not yet lead to services well focused on progam 
participants. For these programs the effectiveness of program services is still 
largely untested. * ... 

Finally, an occasional evaluation in Elementary and secondary education has 
found that participants in the program are not those identified as the needy 
group, with the result that policymakers in the Congress ahd the administration 
have had to clarify eligibility criteria (e.g., ESEA Title VII). Of course, this 
means that both implementation and effectiveness in Title VII need to be 
examined for re-examined) smc£ the characteristics of participants have 
presumably' changed. 

Individual elementary and secondary programs covered An volume II of this report 
offer examples of each type: programs which appear to be targeted 

* appropriately, implemented well, and are effective; those which are well- 
targeted -but need more attention in the area of implementation* and those whose 
target ^roup has been modified* based^on findings of earlier evaluatipn studies. 

'• a 

POSTSECONDARY EDUCATION PROGRAMS. 

The primary goal of the Department of Education's postsecondary education 
programs is to increase educational opportunity. The strategy for achieving this 
4goaI rests * . 

principally on providing financial assistance to both students and institutions. ^ 
Federal student assistance as awarded both directly and indirectly through State, 
agencies and education institutions. In fiscal year 1980 the obligations for ED's 
student financial assistance programs totaled almost $5.2 billiorj. Institutional 
assistance is granted primarily through the Developing, Institutions Program and 
special programs for disadvantaged students. In fiscal yea* 1979, institutional 
assistance amounted to over $400 million. 

■ Participation Rates 

I. * • 

Figures on college enrollment for 18- 24-year olds during the 1970-78 period s 
are presented in table 1* Participation rates for various subgroups tend to be^ 
more volatile than for the age group as a whole.' Over this period, participation 
rates for males have declined while those for females have increased. In 
contrast to more or less constant participation rates for whites, participation 
rates for blacks and Bispani&s have increased markedly. 

•* . 
Ta^le 2 presents the Annual enrollment rates of dependent students since 1974 
categorized by family income (measured in current dollars). Over this 5-year 
period, a clear trend of declining SnroIFmejn rates in the over $15,000 family 
income categories emerges for both males and females. This pattern contrasts 
with the enrollment rateS for dependents from lower income families, which-, 
have remained relatively stable during this period. 
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Table 1 



Total Number and Percentage jof 18- to 24- Year Olds . 
Enrolled In College, By Sex, Race, and Spanish Origin' (In Thousands) 



18-To 24-YearWld 

AGE GROUP J970 1971 1972 1973 1974 1975 - 1976 1977 1978 

' ~. 

All 1/ 

Enrolled 8,163 , 8,671 ■ 8,958 8,878 8,788 9,228 * 9,620 9,285a 9,180 

Total Group 20,632 21,613 22,160 ..22,522 * 22,530 22,95£- 23,380 23,336 23,313 

X of Total 39.6 40.1 40.4 - ,39.4 39.0 4<kr\ 41.1 39.8 39.4 

Male ' . -«t 

Enrolled 4.278 4.576 4.713 , 4,619 4.486 4.6^0 4>91 4,556 4,443* 

Total Group 9,498> 10,071 10,447 10.690 '10.673 10.915 * 11,108 11.T66 11.107 

X of Total 45.0- ' 45.4 . 45.1 »~ 43.2 42.0 42.7 ' 42.2 40.8 4O.0 

* St 

Female * 

Enrolled 3,885. 4,095 -4,245 4.259, 4,302 4,565 4,929 4,729- 4,737 

Total Group 11.134 11,542. 11.713 1K832 / . 11.857 12.044 J2.272 12.170 12,206 

X of Total 34.9 35.5 ."36.2 36.0 ■ 36.3 37.9 40-2 38.9 38.8 

* 

■ 

White , , 

Enrolled 7.481 7,885 8,073 7,976 7,807 8,191 8,454 8,149 7,970 

. Total Group 17,900 18,737 - 19.155 19,332 19,356 19,666 20,003 . 19,912 19,7*4 

X of Total 41.8 42.1 42.1 41.3 40.3 41.7 42.3 40.9 40.3 

Black , ' . 

Enrolled 513 668 753 748 ' 803 883 981 927 971 ' 

. Total Group 2,502 2,637 2,755 ' 2,892 2,828 2,890 2,987 3,038 3,099 

X of Total \^20.5- 25.3 , 27.3 25.9 28.4 « 30.6 32.8 30.5 31.3 

"—-v '/ - 

Spanish Origin 2/ » . I / 

Enrolled "* n.a. *n.a. 214 > 278 35? .368 383 327 332 

Total uruup n.«.s n. d. ' 1.217 1.1GJ 1,380 l',312 1.4GU 1,425 1.44C 

X of Total - n.a\ n.a. 17.6 • 23.5 25.9 28.0 27.3 22.9 22.9 
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y Students of other races ire Included in total for all students but are not listed separately. . , 

y As, attached • ' . / 

• * * * 

SOURCE. Calculated from U.S. Bureau of Census, Current Population Reports , Jerles, P-20, No, 222, Table 14, %^ 
Ho. 241. Table 13; No. 260, Table 12, No*. 272, Z&6, 303, 319. 333, .Table 13. n.a. indicates that 
/ the data 1s' not available. * - 



Table 2 



PERCENTAGE OF 18-TO 24-YEAR-0L0 OEPEMDENT FAMILY MEMBERS ENROLLED IN COLLEGE, - 

BY SEX AND FAMILJf INCOME 



Family Income 

# 

All 

$ • 0- 4,999 
5,000- 9,999' 
10,000-14,999 
15,000-19,999 

,20,000-24,999, 
25,000 +' 



'< 



1974 

13.0 
14.9 
20.7 
30.5 
43.3 
58.8 



Percent 18-24 ENROLL EO 
1975 ' 1976 



14.3 
15. '6 
22.1 
29.2 
41.4 
58.1 



14.5 
15.1 
21.4 
3 2". 8 
44.1 
55.2 



1977 



1J.1 
15.5 
19. 

26 

38.2 
"54.6 



:! 



1978 

13.1 
15.1 
18.4 
23.2 
32.6 
48.1 



Male 

$ 0- 4,999 
5,000- 9,999 
10, 000- 14^9 
15,000-19,999 
20,000^24,999 
2-5,000 + 



16.2, 

18.0 

22.7 

32.2 

44.7 

58.1 



16.7 
'17.7 
24.9 
34.0 
43.8 
56.2 



17.2 
17.1 
22.6 
33.1 
41.8 
52.9 



14.4 
17.4 
20.4 
27.2 

■r 



16.6 
17.2 
19.6 
25.1 
32.9 
47.0 



Female * 

$ 0- 4,999 
5,000- 9,999 
10,000-14,999 
15,000-19,999 
20.000--24.999 
25,000 + 



10.6 
12.4 
T8.8 
28.7 
41.6 
59.7 



14.3 
15.6 
22.1 
29.2 
41.4 \ 
58.1 X . 



12.6 
13.5 
20.4 
32.5 
46 .4 
57.9 



12.1 




10.8 


14.0 




13.4 


18*4- 




'17.3 


25.7 
36'. 3 




21.7 




32.2 


55.3 




49.6 



SOURCE: Calculated from U.S. Bureau of Census, Current Population Reports , Series P-20, 
.Nos. 286, 303, 319, 3.33, 346. . x 
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The patterns do, however, appear to be consbteht with a general decline in the 
, long-run financial benefits accruing to a postsecondary education - a dec '"j£_ 
•that is mitigated for certain population subgroups (minorities and the poor) by^ 
transfer payrT^ents in the form <Jf student financial aid that is conditional upon 
attendance. '* % xl 

The preceding discussion has focused exclusively on college enrollment because 
no'COTTiparable time-series data exist for the noncollegiate sector of 
postsecondary education — proprietary and public technical/vbcatiorial schools* 

Improving Access and Choice Through Student Support * * * 

Enhancing educational opportunity through student .financial aid programs 
removes financial barriers to a postsecondary education for qualified applicants 
(access) and provides each potential student with as wide a variety of 
postsecondary education options as possible (chtfice). A number of assumptions 
underlie this general position. They are: (1) that parents bear the primary 
responsibility for finding their children's education; (2> that students 
themselves bear soiflHp the burden of finaricing their education; (3) that the 
portion of the fmandifburden borne by the students themselves be distributed 
as equitably as possible; and (ft) that the most needy students be aided first, 

• 

ED-administered programs provide, three types of student assistance* Grant or 
non-returnable aid is provided by the Basic Educational Opportunity Grant 
(BEOG) y Supplemental Educational Opportunity Grant (SEOG), and State Student 
Incentive Grant (SSIG) programs. Obligations for these programs totaled $2.8 
billion irf fiscal year 1980. Loans are provided by the Guaranteed Student Loan 
(GSL), National Direct Student Loans (NDSL), and Health Education Assistance 
Loan (HEAL)'programs, The 1980 obligations for these programs amounted to 
$L9 billion. 6 Earnings are provided by the College' Work-Study.(CWS) Program. 
Obligations for CWS were 5550 million in fiscal year 1980. 

The grant programs (as opposed to the loan and work programs) have come to 
serve distinct roles ih removing financial barriers to access and increasing w 
choice. Grants are seen as providing the financial support that less well-off 
parents are unable to provide. Loans and work-study jobs, on the other hand, 
offer students the means to share the financial burden of attending ah 
educational institution. Offering the students both loan and work opportunities 
allows them a choice in the timing of when they ultimately bear their share of 
the financial burden. Work allows them to bear it concurrently with their 
schooling; loans allow them to put off the burden until after they finish their 
education. 

Table 3 displays the proportion of undergraduate students enrolled on at least av 
half-time basis who were aided by one or more of ED's student assistance 
programs (BEOG, SEOG, NDSL, GSL, and CWS?) categorized by dependency . 
status, family income, and category of institution attended. 

In academic year 1978-70, 36 percent of all undergraduate's who were enrolled on 
at least a half-time basis were receiving some student assistance from ED. Of 
those attending 4-year public institutions," 36 percent were receiving aid; of . 
those attending 4-year private institutions, 45 percent were receiving aid; of 
those attending' 2-year public institutions, 26 percent were receiving aid; of 
those attending 2-year private institutions, 37 percent were receiving aid; and of 

* 
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Table 3 



Percentage of Undergraduates Enrolled at Least Half-time Receiving 
Aid from at Least One of ED Student Assistance Programs 
{Unduplicated Count) 1978-79 




By Family Income Group 


Percent 


Dependent 


» $ 0- 5,999 
6,000-11,999 
12.000-17,999 
18,000-24,999 
25,000-29,999 
30,000 or more 


67 
47 
27 
21 
8 


Independent* 




37 


All Students 


# * 


36 



By Institutional Category 

4 -* Year Public 
4 - Year Private 
2 - Year Public 
2 - Year Private 
Proprietary 



36 
45 
26 
37 
63 



tloie: —These include only the Bas.ic Educational Opportunity Grants, 
Supplemental Educational Opportunity Grants, National Direct 
Student Loan, Guaranteed Student Loan and the College Work- 
Study Programs; the State Student Incentive Grants Program is 
not included. 

Source; "Study of Program Management Procedures in the Campus-Based 
and Basis Grant Programs - Stage 2", Applied Management 
.Sciences, Silver Spring, MD; 1980. Contract funded by 
Office of Evaluation and Program Managememt/ED j 



* Considered independent of parental support for student and distribution 
purposes. 



those attending proprietary institutions, 63 percent were receiving aid. Overall, 
a greater proportion of students appear to be aided lh the more* expensive 
^ categories of postsecondary institutions* _ J 

As might be expected, among dependent students the proportion receiving 
student did declined as family income increased* In the lowest income category 
($0 - $5,999), 76 percent of the students received some form of assistance, 
whereas in the highest income category ($30/000 or more) only 8 percent of the 
students received some assistance* Among independent students, 37 percent 
received aid from ED student assistance programs* These percentages are based 
on undup Heated counts of students receiving aid from any of the five ED 
programs. * 

Table % describes the extent to which undergraduates received aid from more 
than one ED program* BEOG recipients were least likely .to receive aid from 
other ED programs; if they did receive other aid, it would most likely be a CW*S 
job. The vast majority of SEOG, CWS, and NDSL recipients also receive BEOG 
assistance* There appears to be very little overlap between the loan programs* 
Only 10 percent of NDSL borrowers are repoVted to also have a guaranteed 
student loan, and only 16 percent of those with a guaranteed student loan are 
reported as having an NDSL* 

Impact of Student Financial Aid Programs 

The achievement of the "access" objective of the student financial assistance 
programs can be measured by the absolute size of the financial burden a student 
must bear if he or she is to attend a postsecondary institution*^ This "student 
burden" is the difference between the cost of attending an institution (tuition, 
fees, room, board, etc*) and the npnreturnable aid the student receives from his 
family, the Government, or other sources (i*s ff family contribution, grants, and 
scholarships)* Presumably*4he student burden is financed by some combination of 
borrowing and w,ork activity, and thus for students attending postsecondary 
institutions, it can be measured by the sum of their loans, earnings from part- 
|time work, and savings fr^m summer work* 

Two aspects of the student burden can be used in measuring the impact of 
student aid programs on "access*" The first is ^he size of the student burden* If 
we assume that the student should be free to choose between current and future 
wbjrk activity, burden size should not exceed reasonable part-time earnings and 
summer savings alone, or reasonable borrowing alone* The second important 
aspect of the student burden as a measure of program impact is the degree to 
which it is equal or unequal across neqd patejjories, as measured by family 
resources, for different typoa^f institutions TselectTveness, public or private, 2 
yeafs or * years). This aspect may be interpreted as measuring the degree of 
equality of financial opportunity, which is derived from the premise that the 
burden borne by students should be as equal as possible* 

The performance^oTthe student financial aid programs with respect to the 
"choice" objective can also be measured in*terms of the student burden* Are all 
postsecondary education options "affordable" in terms of th6 burden a student 
must bear, reg a r d less of family resources? If burdens increase, cjn they still be 
covered by. available loans and work? Relative income equality Is a second 
useful criterion in evaluating financial aid programs with respect to choice* The 
degree of equality in relative burdens serves as a measure of the extent to wh>rii 
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Table 4 

Percentage Rates of Program Overlap 1n Student Aid Recipiency, 
For Undergraduates in the Fall of 1979 ^ 

m 

, • 

o Of students receiving BEOG's, . ' ' . 

\ 24% received SEOG's, 
36% received CWS, 

25% .received HDSL's, " - 

10% received GSL's. 4 

. 

o Of students receiving SEOG's, * : 

' 87% received BEOG's, 

50% received CWS, 

48% 'received HDSL's, r 
' 8% received GSL's. 

* 

o Of stents receiving CWS, 

82% received BEOG's, 
37% received SEOG's, 
35% received NDSL's, 
14% received GSL's. 

o Of students receiving HDSL's, 

76% received BEOG's, 
40% received SEOG's, 
38% received CWS, 
-* 10% 'received GSL's. 

' * 

o Of students receiving GSL's, 

" 48% received BEOG's, 
11% received SEOG's, ' 
26%" received CWS, 
~* . 16% received ND/L's; 

* 

Source: "Study of the Impact of the Middle Income Student Assistance 
Act," Applied Management Sciences, Silver Spring, MD, 1980. 
Contract funded by Office of Evaluation and Program Managemejjt/ED 
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grant programs equalize the financial te/ms upon which prtepective'students 
must choose between postjecondary options/ Clearly r relanve student burdens 
will be equal if the student burden associated with each education option is the * 
.same regardless of the student's family resources. An education option in this 
context c|in be defined by the total co^t of attendance or the student's expense 
budget. 

Evidence ^oncerning the Impact ofrED's^ Student Financiai Assistance Programs 

Tables 5 and 6 show estimates of the mean expected family (contribution, totA * 
grant aid received, and student burden borne by financial aid applicants, 

«tegorized by dependency status, family income for dependent students, and the f 
st (table 5) or type of the institution attended (table 6). The estimates of 
mean Student burden are useful in assessing the impact of Ep's financial 
assistance programs. It should, howeveY, be dfcmpmbered that ttese ^estimates 
are derived f rom>data on aid applicants only.' 

A comparison of student ULrdens with what students can be reasonably expected 

40 earn oj borrow sheds light on the achievement of both the access and choice 

objectives of the student financial aid programs. As a reference,^ student 

workir\g affthe minimum wage for 1.5 hours a week during the school^ar would 

earn approximately $1,350. Summer work could reasonably produce arother^500 

in savings. Therefore, from work alone a student should be able to finance a 

student burden of $1,850, provided, of course, that*a job exists. Under the 

Guaranteed Student Loan Program v all students are eligible to borrow up to 

$1,500 per year -for 3 years; "needy" students can be afforded loans by 

postsecondary institutions of up to $2,500 in their first, 2 xears of study and 

another $2,500 in their second 2 years under the National Direct Student Loan 

Program. k • ? 

. $ 

If access is interpreted as adequate financial resources to attend a 2-year public 
institution, the mean student burdens presented in table 6 indicate that the 
access objective has been Veasonably accomplished. For |dl dependent students, 
the student burden coulcl be easily financed by part-time worff alone. The 
independent students' burdens are significantly higher but seem manageable if 
students commit much more time to work activities or are willing to take out 
sizable loans. Little change is noted if access is defined in terms of 4- year 
public institutions. Student burdens*at .these schools'are higher for all 
categories, except independent students. Where higher, still easily financed by 
s WbrJ^activity. 

Wt\at # about choice? How feasible are the student Uurdens at private and 
proprietary schools? Looking at table 6, we see that the largest mean student 
burden is $3,774 (for independent students at private 4-year institutions). A 
bctfden of this magnitude is approximately $300 less than the sum of reasonable^ 
expected student earnings ($1,850) and annual student borrowing ($2,500). Where 
postsecondary education options are defined by their total cost in table 5, the 
largest mean student burden is $5,349 (for independent students at institutions 
costing ovejv$6,000). A^lburden this large is almost $1,000 more than the sum of 
#hat.A student cap boirow arjjd reasonably be expected to earn ($4,350). If these 
^students chtfose to attend a fiigh-cost private institution, they can do so only at 
great personal sacrifice. * . , 



Table 5 



Mean Enacted F#*tly Contribution Grant Aid eft Student Burden for Aid Applicants by Dependency 
Status, by Fwlv inwe. end by the 5t«<tent Expense Byd?et gf the Institution Being Attended 1979-^0 

(In dollars) * 



Dependent Students by Faatly Income 



In dependent Students 



Student Expanse , 
Sudoet ^ * i 


0 

-S.999 * 


$6,000 
.11.999 


S12.0&0 
-17,999 


$18,000 
•23,999 


S24.000 
-29,999 


(30.000 
4 er*J Over 




0- 


Expected Family Contribution 




277 


625 


782 


1,293 


1.033 


.V£6 


v .S2.5O0 


• All Grants 

• 


1.097 


1.063 


829 


In 


680 


76$ 


649 * 




tt«uf*At flu rden 


510 


S3 5 


574 


sfo 


287 


250 


" 880 


■ S?>501 - 


Expected Fanlly Contribution 


439 


* 

629 


661 


1,016 
% 


1,171 > 


• *.447 


293 


3,005 


All Grants. 


1.230 


1.096 


1.082 


862 






727 




Student burden 




1.021 


1 ,U£V 

* 1 


W 7 


£31 


% 808 


1,772 


* J »w l » t 


, . . - — -jj- 

f ■'W't^ r lag 4 1 V r AMt «4Kkbt 4 Afl 

cx^meo '#™»ijr ^omr«Pvnvn 


302 


370 




729 


1,151 


1,476 


1,645 


225 




Aai i»r4ni* 


1.786 


1.670 




1.201 


823 


1.018 


948 




Student Burden 


1,0 S3 


1,573 


1.414 


1.229 


1,314 


959 , 


2,388 


$4,001 - 


a) 

Expected Fa»11y Contribution 


2S7 


m 


767 


1.779 


2,665 


fc . 

2.226 


382 


* 5.000 


AVLCrants » * 


2.260 


2,271 


1,880 


1.238 




1.00? % 


1.288 




i 

Student Burden 
* ♦ 


2.037 


1,977 


J. 902 


1.565 


1J25 


1,348 


2,841 


• 

SS.001 - 


1 e) 
Expected Faulty Contribution 


369 * 


$45 


884 


1.405 


2,026 


3,146* m 


361 


^ 6.000 


• All Grants 


2 612 
m 


2.40$ 


2.002 


1 904 


1.634 


979 


1 424 


* 


» Student 8urden v 
— , - - ■ 


2,437 


2.59^ 


2,648 


2,251 


1.960 


J.396 


3.642 
— , — i 


Over 


a) 

Expected Family Contribution 


860 


823 


1.379 * 


2.45ft 


3,012 m 4,¥* 


893 


6.000 


All Crants m 


3*293 


3.366 


2.844 
• 


2 61*1 


2.25$ 


1.421 


1.515 


t 

* 


Student Burden 


3.222 


3,186 


3.413 


2.285 




1,626 


5.349 


Note, *} The Expected Fa«Uy Contribution 1* coftputetf accoroTng to the Jnffom 

n not to be less then S1.1O0 for"* student living with parents. 

* 


flethodoTogy 


. adjuited^Sfiere appropriate so - * 


* * ' b) Cell frequency fewer then SO* / 










• 






Source 

* 


Study of the Impact of the Kiddle-lnco** Student Assistance Act/ 1 Applied Management Sciences. Silver tyring. MD 
1980* Contract funded by Office o\ Evaluation and Prolan Hane9**ent/E0, 
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He Jit Expected F*»i1/ Contr i but 1 0ft. Grant Aid and Student fyrden for Aid Xpplicants by Dependency 
Status, by Family Intone, and by Type of Institution Beirwj Attended 1979-90 

Tin doners) 





• 

•* 




dependent Students by Family I new 


Independent Students 


Type of 
Institution 


\ 

y — 0 


S6-000 
.11,999 


4 

$12,000 
-.17,999 


518,000 
-23,999 


S24 000 
-29,999 


WO. 000 
and over 


. 


4 Yt*r 


Expected Family Contribution 


173 


361 


666 


1,592 


1.774 


2,332 ■ 


414 


Public 


All Grant* 


1,600 


1.499 


1.388 


1,078 


744 


866 


1,097 




Student fturden 


i.i$8 


1,418 


1.686 


1.215 


1*366 


1,067 


2.784 




4) 

Expected Fa*lly Contribution 


m 




1,109 , 


1,670 


2,476 


3,9J6 


615 


Private 


All Grants 


2,999 


2.829 


2,4# 


2.347 


1.944 


1.362 


1.631 




Student 8urden 


2.089 


2.341 


2.407 


2,036 


'1,887 


1,680 


3.744 


2 rear 


4) 

Expected Family Contribution 
All Grants 

* * 


1,085 


1,078 


761 
1.033 


1,126 
792* 


1,*97 

690 


1,987,, 
686 


* 349, 
978- 


* 


Student Burden" 


1.086 


1.061 


926 


986 


786 


918 


2.848 


2 Tear » 


a) 

Expected Family Contribution 
• 


69$ 


l 

469 


607 


1.122 


2.084 


4,1^ 


409 . 


Private 


Atl Grants 


1.6M 


1,771 


1 377 


1,138 


1,024 


860 






Student porden 




1.282 


1,749 


1.227 


.932 


* #8 


1,978 


Proprietary 


Expected family Contribution 




664 


\ 006 


- 1 201 


1.611 


2.176 * ■ 


> 

. 269 




AM Grants 


1.846 


1.789 


1.362 


1,092 


844 


+06 


1,342 




* Student 8urden 


- 1,968 


1.806 


1.691 


2.161 


2 440* 


2,064 


7.733 



Kote aj The Expected Family Contribution »s> computed according to the Onifwn HethodoJogy. - adjusted where appropriate so 
as not to be less thin Si. 100 for a student living with parents. * * 

b) Cell, frequency fewer tban 30, . 

t 

Source "Study of the t«np^ct of the Middle income student A*s.sT*nce Act 1 , Apphed Management Sciences, Silver Spring, ft) 
1980, Contract funded by Office of Evaluation and Program Kajaget^flt/ED. 




Tables 5 and 6 present only rtfean student burdens* a fuller picture of the 
affordability of student burdens relative to what students can either earn or 
borrow is presented in table 7. This table shows the results of a comparison of 
individual student burdens with the amounts students could be expected to earn 
fropi part-time work ($1,350), earn doing part-time work combined with savings 
from summer work ($1,850), or borrow ($2,500). 9 The table indicates that all of 
the students attending the lowest cost instutitions ($0>2 f 500), presumably public 
2-year and 4-year institutions, could have financed their student burdens with 
the subsidized GSL's or NDSL?s, antf 93 percent could have financed their student 
burdens with work alone. Moving to the next most expensive category of 
institution ($2^01-3,000), 92~percent of the students could have.financed their 
student burden with a subsidized loan alone, and 82 percent could have financed, 
it by work alone. It is reasonable to assume that ar\ accessible institution near 
every student falls within thpseJirst two budget categories, supporting the 
conclusion that access, or fnore specifically financial access, has nearly been 



acttieved. 



table 7 shows that as institutional costs become greater, the percentages of 
students whose student burdens exceed allowable annual loan limits and 
reasonable ievgl^jjf part-tinte earnings increase rapidly. Nonetheless, students 
still attend these institutions, making it reasonable to* assume that students who 
are managing to enroll at more expensive institutions could also have attended 
less expensive ones. On ttye basis of what is presented in this taj>le and^in the 
two precedin&tatyes, a considerable degree 6i real choice among postsecondary 
iristitutions #cists for ipost >tudent$, .But for many students choice comes only 
at the prijept m\idyAig*i^^^»npl sacrifice in terms of student* burdens, 
particularly ^qjvjhe {ftdep'Sp&&t\student. 




Turning to the question of tfliether 'there is equality in the financial terms of 
access and ?WipeaTwng^tud^^K^ith different family resources, we can look 
at table 5 ajpip/ A^compferison of mean stOdent burdens for differing 
dependency ancfrfemUy i/coine categories within institutional cost categories 
indicates equally dtqarfg dependent students whose family intomes are less than 
$ft,000. fteap student burdens tend to decline for dependent students as they 
% move to higher intome categories and increase markedly for independent 

* students. „ * 

Table S provides another perspective on how the distribution of grant aid af fectjr 
the*degree of equalitjf in the financial ternjs of access and choice. In this taWe, 
the hypotheti<^l student burden in the absence of grant aid as a fraction of ine 

h student's expense budget is compared with the actual student burden as a / 
f rattion of that same budget. Each cell shows both the mean and standa^Q 
devotion (a measure of dispersion about the mean) of the hypotheticaj^Cnd 
acti&il fractions for students ii^the cdlU In all c^ses grant aid gre^rty reduces 
the percentage of a student's budget that must be financed wit^roans, and/or 
work; the manner in which grants are distributed increases^He inequality of the 
percentage for most Categories iif^hiih nl i "*Tli|,i iimn rtiin the inequality of 

^student burden as a percent ||n niipi Mil I u l| i I i most pronounced in the 
I tCflaJj i*i MJiy III In IkiiIiihi ($0-2*500) and in all institutional categories 

# fo^MZfependcnt students. \ ^ v *» 

JifK conclusion, it appears from the data available that ED's student financial aid 
programs have been reasonably successful in providing financial access to some 
~g 'evei of postsecondary education for all prospective students with financial need 
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Table 7 
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The Proportion of 8000 StM<5ent A;d Applicents Whose Stwtent Burden £*cte<Js $1 3SC 


$1,850 and 52,500 respectively 






V 


Oeoendei* Studeat by faally JrcofY 


* 


Independent 
Students 


All 

■ Students 


Bu*9tt 


* 


0 

-6,999 


$ 6,000 
-11/399 


$12,000 
•17,999 


$18,000 
•23,999 


$24,000 
•29.999 


$30,000 
and Q*tr 






0- 


Greater Thjn $1*350 


:o9 


.07 


.09 


-.07 


.03 


0 


.29 


.15 





j. 15. i»re*ier ifwn 

Crtmr Thjn $2,600 


■Ul 

0 


0 


.04 

0 


.03 

; 0 


0 

« 

0 


™ 0 

• a) 
0 


.18 

0 


.07 

.00 


52.501- 


f 

S.4. Grtlter Thjn 5U350 


.37 


.26 


.24 


■ 13 


.19 


.04 " 


.74 


.39 


3,000 


5. ft. Cr#JtCr Thin 51 8^0 

S.B« totter Than $2,600 


.02 


.05 
.01 


.04 
.01 


s02 
0 


.02 
.02 


• Cl 

a) 

0 


.26 


.08 


$ 3.000: 


S.ft. Greater Than $1,350 


.64 


.66 


.64 


.43 


.50 


.29 


.88 


1 „ 

\ .42 ^ 


4,000 f 


S.B-. fretter Than $1,850 


.29 


.28 


.21 


.19 


.20 


.25 


.76 


J 


$.8. greater Than 52,500 


.08 


.06 


.03 


.07 


.09 


,17 


.52 * 


.22 


£4,001- 


5.ft. greater Then $1,350 


.^3 




.72 


.56 


.38 


.48 


.90 


.77 


Vooo 


S.8. Greater Than $1,860 


.5? 


,68 


.53 


.47 


.27 


.41 




.67 




S.B. Greater Then tf.WI 


# .31 


.24 


.23 


.21 


.15 


.16 


.69 


.46 . 





— 1 ttaJSreater Then Jl,360 


,91 

* 


.92 


.92 


.30 


.7b 


.68 




.86 


b.UW 


5 ifi^mTtirWi^ 




• 


JO 




.66 


.38 


.91 


.77 




"reater *fhan $?,50O 


V 41 


.51 


.64 


.43 


,4m 


• U 


<.83 


.66 


Over 


S.** greater Then $1,350 




.93 


.92 


.75 


.7<r^ 


.55 


.91 


*« 


6,000 


$.». Greater Than $1,850 


.76 


\ .81 
1 * 


.83 


.64 . 


.66. 


.40 


.90 


,76 




$.A, Greater Than $2,500 
* 


.63 


J .65 


.63 


,50 


,47 


,32 


.83 


.66 



*ote: |) Cell frequency fever than 30 

Swrce "Study of t*e impact of the Middle income Student Assistance Act ', Applied *ana9e«eflt Sciences, Silver spring, M), 
1980, Contract funded by Office of Evaluation and Program K*na9e*ent/En 
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Table 8 



on the Level and Variation In Student Burden as a Proportion of Student 



Dependent Student By Fjglly Inccgg 
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.86 


.20 


.69 


.22 


.64 


.25 


.44 


.27 


.49 


•33 


.88 


.27" 


ft? 
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■ (J 




b) c) 






















d) 


M 


.SO 


.33 


.29 


$2,500 


With Grants 


.24 


ao 


.21 


.41 


.09. 


.09 


.13 


.17 


J8 


.25 




w.thowt vranvs 


\ r >* * If 


.77 


.22 


.76 


M 


.63 


-20 


.5? 




.45 


.15 

d) 


.69 


. B 2a 


.78 




3*000 


With Grants 


.34 .22 


.37 


.19 


.37 




.32 


.16 


.32 


.21 


.30 


.13 


.f3 


.34 


M 


.25 


$3,001- 


Without Grants 


.91 .12 


.90 


.13 


.80 


.20 


.68 


.21 




.24 


.55 


.31 


.9* 


.16 


.84 


.21 


4,000 


With Grants 


.4? .20 


.44 




.39 


.17 


.34 


.24 


.36 


.28- 


.27 


.38 


.67 


.29 


.50 


.26 


$4,001* 


Without Grants 


.94 .25 


.93 


.10 


.83 


.19 


.61 


.32 


■<H 


.33 


.51 


.31 


.92 


.20 


.83 


.27 


' 5.000 


With Grants 


.45 .10 


.43 


.19 


.42 


.20 


.34 


.27 


.25 




.20 


.35 


.63 


.2$ 


.51 


.27 


$5,001- 


WUbovt Grants 


,-93 .14 


.90 


.14 


.$4 


.14 


.75 


.21 


.64 


.24 


.43 


.28 


.93 


.18 


.82 


.25 


6,000 


With Grants 


.45 .17 


.47 


.1$ 


.48 


.19 


.40 


.20 


.35 


.27 


.25 


.24 


.67 


.25 


.50 


.25 


Over 


Without Grants* , 


.88 .20 


.$9 


.20 


.82 


.19 


'.68 


.28 


.60 


.29 


.40 


.28 


.89 


.26 


.76 


.31 


$*,ooo* 


With Grants 


.44 .24 


.43 


.22 




.22 


.32 


.20 


.30 


.25 ' 


.21 


.24 


.68 


.28 


.49 


.30 



Hote* «} Student rerden Without Grants as a Percent of Bod^et ■ {6u**** - Ftt*cted family Contribution! - Budget 4 

b) Student Burden With Grants as a Percent of fcudget - (Budget - Expected Family Contribut Ton) - Budget 

c) The Expected Family Contribution is ccnputed according to the Onjfora "tethodology. adjusted where appropriate 
so as not to be less than»$l,100 for a student living with parents. 

d) Cell frequenot less than 30 

e) SO - Statfy^Oevlatlon m 

Source. "Study of the Middle Income Student Assistance Act"* Applied Management Science*! Silver Spring, M>, 1930. 
Contract funded by the Office of Evaluation and Pro?** «4na9«*ent/£n .. 
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These programs appear to.have made even the most expensive postsecondary 
t educatio n opti ons potentially affordable even for the poorest students* But 
attendance t atTliglvcost institutions seems to come only at the price of very high 
personal sacrifice for certain categories of students, particularly the 
independent student* ' „ / 

The MioUie Income Student Assistance Act (PiflSAA) became law on November 1. 
1978. However, its provision affecting the BEOG, SEOG, and CWS programs did 
not take effect until the start of the 1979*80 academic year. The intent of this 
legislation was to reduce the student burden of middle-income students* To see 
if this occurred, one can compare the entries in this repoFt!s.table 5 with those in 
a similar table in the 1979 Annual Evaluation Report *^ Table presents this 
comparison in terms of the 1978-79 to 197^-80 change in each item. • * 

What is the net effect of these changes? If we divide the dependent students 
between low and middle income at $12,000 f^ schools that cost students up tg 
$5,000, and at $18,000 for more expensive schools, we find that student burdens 
> increased in 12 oyt of 1^ low-income combmatfcns, whereas student burdens 
decreased in 20 out of 22 nr^ die-income combinations* This pattern of changes 
is highly significant statistically J ' Thus, M1SAA appears to have had its 
intended effect with respect to the middle-income students uPthat the mean 
burden appears to have been more clearly equalized for a number of budget . 
groups* To a large extent, this equalization has been accomplished through an 
increase in the mean grant level* However, in a number of these groups there 
was a drop in the mean family contribution as well* Although at first glance, it 
may have appeared that M1SAA ^robbed Peter to pay Paul," in reality itlias , 
provided for the equalization of mean student burden levels* ^ 
* ■ 

Student Persistence t 

As noted in earlier Annual Evaluation Reports, past studies revealed that 70 
percent of all entering freshmen eventually graduate from college* Fifty 
percent of this group graduated from the institution at which they started; the 
others graduated after transferring to other institutions. Furthermore, 30 
percent of the students who transferred did so in their second year* 

Analysis of recent data collected an ED study on student persisteno^flfditates 
that financial aid is related to Student persistence in a positive manner* \nf 
particular, those students with a) grant or a loan above $1,000 were/found/fnore 
likely to persist. Financial aid, is might be expected, assists the Ipw-in&me 
student more than a studentJrom higher income families* Work aid ajfpears to 
enhance the persistence of low-income students especially if they receive no 
grant aid* Work aid, howjever, does not appear to be a factor lor highej income 
students especially if they already have a grant* • 

* 

Improving Access, Choice, and Quality Through Support of jnsthutions ^ 

The Department of Eduction also administers programs that grant funds 
directly to the institution* Programs such as these allow the institution to. , 
maintain and improve the quantity and quality \f the educational services it . 
provides students without passing the full cost of these services on to students* 
These programs, like the student aid programs, also help the institution Reduce or 
hold constant thcf burden to students in a period of rising costs by reducing the ♦ 
net cost of instruction* x , " 
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A Comparison Of Changes <n MUr Amounts of Family Contributor* Grant *1d And 
Ihe Student 8urden (WortAoans) Fro* 1978-79 to 1979-80 by Family lnv»e Sy Student BudQe* a/ 
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'Study of the Impact ofVse Middle Incone Student Assistance Act', Applied Kana9e*ent Sciences, 
Silver Spring, K) 1980, Contract funded by the„0ffjce of evaluation and Proyrm *ana9e«ent/E0 

v ' 

aj The purpose of this table is to's*w ho* ptssao* of MISAA affected key *injnc-«l components of student - 
resources by faniiy income, *o adjustment was made for inflation of family income and itudent budsets. 



Federal institutional aid programs administered by ED make up about 7 percent 
of the total ED higher education budget ($3Z0 million in fiscal year 1980 of a 
total of $5.3 bJllion).12 title HI of the Higher Education Act (HEA), the 
Developing Institutions Program, accounts for a little less than one-third of this 
total ($110 million in fiscal year 1930). Title III promotes detailed institutional 
.program planning, curriculum development, faculty development, and improved 
administrative practices by providing resources to help participating institutions 
finance these programs* Generally, .t he focus has been on institutions that serve 
large numbers o^ disadvantaged students. This focus is consistent wjdh the larger 
goal of Federal aid for postsecondary programs; to enhance equal educational 
opportunity. HEA- Title HI program eligibility criteria have included as -a^key 
factor the relative number of disadvantaged students the institution serves. 
Thus, the program* can be said to be on target if the relative number of** 
disadvantaged served in developing institutions exceeds those in the student 
population a$ a whole. BEOG recipierjfc, the economically disadvantaged 
students, on the average made up 28.5 percent of the student body in HEA- Title 
III irtstitutio'ns compared to 17.9 percent for other 2- and 4-year colleges in 
academic year 19£7-£$. / 

Recent program participants provide the only available evidence on program , 
effectiveness. Few participating institutions have closed since the inception ok^ 

" the program. Given that access and choice are Federal goals and since these , ^ 
institutions serve large numbers of low-income students^ the closing of any , w 
institution would reduce the number and choice of postsecondary education 
institutions available for students enrolled in the affected institutions.^ 
Evidepca shows that proximity to public and private nonselective colleges 
influences college entrance,, and that low-income students tend to live at home 

^hUe^attendmg college. TWb distance from home to college is strongly and 
consistently associated with parental income even after other student 
characteristics such as parefrtal education, ability, and race have been taken into 
account.!* Thus, certainly choice and, to a lesser degree, access are reduced 
with the closing of any institution. " • 

tfhile student financial assistance programs can be judged,in terms of their 
impact on either removing or equalizing financial barriers for students, other 
Federal programs are aimed at assisting the student by removing nonf inancial 
barriers to access and choice. Target groups foMhese programs are 
disadvantaged persons with academic promise wro lack adequate academic 
preparation or who are insufficientlytmotivaxed. 

The programs attempt to identify potential students and provide them with 
counseling and remedial assistance to encourage therp to enter and persist in 
postsecondary education. Present programs focus ^ students both at the 
precollege and college levels* however, current funding levels allow for serving 
no more than one in seven students'who "might otherwise qualify. 

The recently completed stuffy of the Upward Bound Program identified 
substantial impact upon students' entry into and persistence in postsecondary 
education programs. Currently, a major study of the Special Services for 
Disadvantaged Students Program is in progress. It is, however, too early to 
report on the impact of the Special Services Program. More detail.on each study) 
is presented in the program section of volume II alortfc syith program ar.d study 
information on the Talent Searched Educational Opportunity Center Programs. 
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tn addition y the Departmept of Education administers a number of small 1 
categorical and grant aid programs ($50 million or less in annual appropriations). 
Included are fellowship programs, construction assistance, and international 
education. These programs are also discussed in detail in volun}* H. f \ 
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SPECIAL CATEGORY PROGRAMS 
.Occupational, Adult, and Career Education 



\ 



The major special category programs are authorized by the Vocational Education 
Act (VEA), the Adult Education Act (AEA), and the Career Education Incentive 
Act (CEIA) which, with the exception pf the CEIA, are administered by the 
Office of Vocational and Adult Education. The CEIA is administered by the 
Office of Elementary and Secondary Education. 

By far the largest source of Federal support for these special category 
is allocated to vocational education programs. The purpose of vo< 
education is to help individuals bridge the gap between education and\r6rk by 
providing the knowledge and training needed«for specific occupations. Currently, 
funding for Federal vocational education programs"I?of two basic types: 




Formula Grants to Statas 



Allocated on a formula basis, "these grants assist States to maintain, 

/\ extend, and improve existing programs of vocational education so 

that persons of all ages wi{l have reafy access to high quality 

vocational training or retraining; to develop new programs of 

vocational education; and to provide part-time employment for. 

youths who need the earnings from such employment to continue 

* their vocational training dh a full-time basis, 
• * 

Funds are distributed through three separate authorities: (1) 
Programs Jpr Disadvantaged; (2) Basic Grants to States; and (3) 
Consumer and Homematcing Education. The largest is Basic Grants 
($562,266,000 in FY 1980). * * 

o Discretionary Programs 

Jhree programs award grants and/or contracts the discretion oi 
the Secretary; Programs of National Significance (research, 
demonstration, and development projects); the Program for Indian 
Tribes and Organizations;*and the Bilingual Vocational Training 
Program. * 

Because of a sevenfold rise in public outlays for vocational education during the 
last decade, more than one-third of the Nation's high schoof students now take 
vocational subjects. While it is true that initial Federal expenditures for 
_ vocational education programs in 1963 gave impetus to State and local spending 
for these^programs, no studies haye been conducted to determine whether the 
Federal program's impact on State/local spending continues. Though it is widel> 
believed that Federal dollars no longer stimulate the expenditure of State and — 
local funds because of the large State and local overmatch, Federal set-asides 
Q*~>r national priority groups do have some catalytic effect. For example, set* 
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asides for the disadvantaged and handicapped have increased the number of 
"^programs for these groups* No evidence is available to show whether the 
postsecondary set-aside is responsible for the doubling of postsecondary 
vocational enrollments (e*g*, in Community colleges) since 1970* 

Measurement problems and interpretation ambiguities nqake it difficult to 
determine whether or not vocational education has been successful* Onef 
problem is that employment is not the primary objective of ail vocational 
students* Another is that economic conditions probably much more powerfully 
influence employment choic^ among youth more than does curriculum choice* 
Despite these limitations) the data from four national longitudinal studies 
provide a sufficiently long timeframe to assess employment outcomes for 
vocational education graduates*'^ 

o Taken together, the longitudinal studies suggest that most secondary 
vocational education have no labor market advantages* The 
exceptions are the young women trained in the office occupations* 
t Compared to their male counterparts and other 1 vocational education 
graduates, they experience fewer periods of unemployment, have 
higher hourly wages, and work in jobs related to their training* Other 
high school vocational get jobs not too unlike nonvocatior\al 
graduates within-3*years after graduating; experience similar rates of 
pay and unemployment; have similar degrees of job satisfaction; are 
no more knowledgeable about occupations than nonvocational 
students. Vocational education students are as likely to drop out as 
other students; are less likely than other graduates to continue 
formal education in colleges and universities but will most likely seek 
additional occupational training. Postschool training (outside of 
colleges) pays off more for vocational graduates than for other 
graduates* - 4 

o Because secondar^ocational students have lower aptitude test 
scores and tend to come from* poorer families than other Students, 
it is often assumed that these students^TouId lose interest in school if 
vocational education were not available* This hypothesis, however, 
has never been tested* 

4 

o Other studies of postsecondary students indicate they are more likely 
to enter training-related jobsr- 

The Vocational Education Act supports programs that train persons in specific 
occupational skills and is primarily focused on training at the secondary and 
postsecondary levels* The career education legislation emphasis is on building a 
broad understanding of different types of work at the elementary4^el with later 
development of decisionmaking skills in deciding on appropriate occupations. 
The Career Education Incentive Act (Public Law 95-207), passed in December 
1977, authojftes formula grants to assist States in implementing career education 
mjocal schools* Jt^also continues the Secretary's discretionary authority to 
make demonstration grants^io State and local education agencies and to 
postsecondary institutions a provision not yet implemented* 

The Adult Education Program focuses on yet another aspect of the transition 
from school to the world of work* Specifically, its legislation provides for 
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support to the States for projects which will assist adults In obtaining the basic 
sWils (i.e., language and computation skills) necessary to function effectively ia 
an occupation. 



Programs for the Handicapped 



The several programs authorized under the Education of the Handicapped Act 
(EHA) have one or more of fou^ basic purposes: (1) provision of direct services^ 
(2) development, demonstration, and dissemination of nev* technologies, teaching 
method^ and materials; (3) training regular and special education personnel, and 
(4) program evaluation. In each program, the role of the Federal Government 
has been stimulative, whereby it provides "seed" money to States and other ^ 
grantees to stimulate increased quantity and quality in all services and to assure 
that program benefits reach previously unserved handicapped children. The 
strategy for evaluating programs for the handicapped is to determine if they 
have accomplished their specific purposes and, more generally, have had the 
desiced stimulative effect. Accordingly, evaluation studies have been of twp 
kinds: (l) those designed to obtain objective data on the impact and 
effectiveness of specified programs,. particularly those which represented a 
major Federal investment of funds; and (2) those designed to provide policy- 
relevant planning information to enable the Federal: Government to target its 
resources more effectively. 

Studies of the first type found that, in general, these programs have 
accomplished their specific purposes. Efforts to isolate the stimulative effect 
and to demonstrate a causal relationship attributable to Federal programs have 
been complicated by factors outside the control of Federal evaluators and 
i program managers. Examples of these factors range from effective lobbying by 
parents and professional groups with special interests in education of the 
handicapped to court cases which have demonstrated that handicapped children 
have not had equal access to educational opportunities. Because of these events, 
an increase in resources for handicapped children has led to a corresponding 
gradual increase in the number of handicapped children receiving services. As 
noted above, the degree to whictvFederal programs have contributed to this 
increase has-not been clearly determined. Furthermore, attempts to 
demonstrate this effect have been complicated by recent legislation which has 
resulted in a significant redefinition of the Federal role in education of the 
handicapped. 

The Education for All Handicapped Children Act of 1975 (PX. 9*M42) amended 
EjJIA, P^trt B, in the following ways: 

o It explicity states that Federal assistance ,to States is to assure 
— access to a free, appropriate public education for all handicapped 
children. 

o It specifies that the unserved have first priority and that 

the most severely handicapped within each disability category 
who are not receiving an adequate education have second 
priority for services relative to all handicapped children. 

o _ It indicates that handicapped children should bfe served in the 
least restrictive environment consistent with their needs and 
® ~ abilities. 
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o It specifies that each child shall have tj^e-benefit of an 

individualized educational program which will be updated at 
least annually* 

■ 

The Office of Special Education and Rehabilitation Services has data x which show 
progress in achieving these legislative objectives* For examples 

t- o Almost 77 percent of the Nation's handicapped school-age children 
are receiving special education and related services today compared 
to less than 50 percent at the tlmaJPJL. 94^142 was enacted, 

o In the annual child count for the school year 1979-80, 46 of the 58 
States and Territories (79 percent) reported an increase over 1978-79 
in the Wumber of handicapped children receiving special education 
anjjjrelated services. 

o Since the passage of P.L. 94-142, over 317,000 additional handicapped 
children have been reported to be receiving special education which 
has required the hiring or reassignment of approximately 19,000 
teachers. 

o The numtfehof preschool children ages 3 through 5 

receiving special education has increased by more than 36,000 v 
in the past * years, a growth of more than 10 percent* 

o Since the enactment of P.L. 94-142 in 1975, the number of 

previously institutionalized handicapped children being served 
by local districts has increased by almost 61 percent. 

Developmental Programs in Education 

A major purpose of several programs administered by ED is to improve the 
Quality of educational practices. These programs are: 



AH of these programs support demonstration projects. Because of limited 
evaluatiop*funds, however, only three programs have been evaluated: 
Community Education, Teacher Corps, the Right-To-Read portion of Basic Skills 
Improvement. j t 

Community Education _ * ■ 

The purposes of this program are to demonstrate effective practices in 
community education and to stimulate wide adoption of these practices. A ' 
recent study indicates that the program has been quite effective in achieving its 
purposes. For example: * 

o Substantial levels of effort (e.g., personnel, material support, and 
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Community Education 
Consumer Education 
Education in the Arts 



Basic Skills Improvement 



Indian Education, Part B 
Indian Education, Part C 
Metric-Education 
Teacher Corps 
Teacher Centfers 




These level* of program effort and activity were achieved, 
in palrt, because there Wfre substantial commitments of 
non-Federal resources supplementing the Federal funds* 

o The majority of the projects which had received Federal 
funds f of only one year were contini^pg at least some of 
these activities and in several, the level %t effort 
even without Federal funds had increased* 

r 

o These project-supported activities had produced desired 
effects on the educational systems and broader community 
* and also produced identifiable changes in individuals which 

were appreciated by the project participants./ 

The primary negative finding was that recordkeeping at the project level was 
inadequate. Routine, program matically meaningful data do not now exist. 

Teacher Corps Training 

Teacher Corps is- currently implementing a new program whereby its field 
projects in partnership with an institute of higher education (IHE), a school 
district, and a community advisory group, provide mservice training for teachers 
that is responsive to both j/istrict and community needs, with a hopetf-for side 
benefit of reforming IHETeaching practices. Additionally, the traimpg is 
intended to instill a greater sensitivity to the needs of minority and/or 
disadvantaged children so that the teacher can work more effectively with these 
children. Teacher Corjfc expects to demonstrate several program models- 
directed t<rivard these objectives. ^ 

*> 

A study of Teacher Corps graduates of preservice training projectsshowed ^ 
partial success in achieving similar objectives. Teacher Corps graduates were 
most different from control group teachers in (l) developing ethnically relevant 
curricula;.(2) using community resources in teaching and for contacting parents; 
and (3) having positive attitudes about reading development and causes of s 
povertyMn the society. ' . 

• These findings reflect Teachep Corps' special concern about low-income minority 
group children. However, teacher performance measures in the study showed no 
differences in such categories as teachers being a change agent in the school or 
in interaction between teacher and pupils in the classroom. Furthef, no 
significant differences between Teacher Corps and control group classes were 
revealed on any reading measure, ^spite a greater emphasis on reading 
instruction and academic 'subjects by control group teachers in grades 2-3. 
Teacher Corps graduates, however, bright **bout significantly greater changes 
in a child's self-concept than those brought about by control group teachers. 
These changes consisted of observed expressions of greater happiness and greater 
self- worth in the classroom, and better scores on important subscales of the 
Piers*Harris self-concept scale* 

Basic Skills* Improvement — Right-To-Read 

■» . * 

This new program extends the scope of the Right-To-Read program, with 
increased responsibilities fo'r improving written and oral communication skills, 
0 *nd mathematics and reading skills. The transition from Right- To-Read to Basic 
£j^£;iH$ Improvement took place in October,l979#* <- I 
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Case studies of. the Right-lVRead State LesyJership Program indicate that tmT' 
State C9mponent£ of the^Right=T£-Rearf program, with a modeSt amount of 
mcjney, appears to have been successful. For example: 

» r . * 

o State Right-to*Read programs have made considerable efforts 
- t9 involve Iclrge nurpbers of local districts in Right-Tp- 
Read efforts. . ' w > 



o 9 training has bejen a major focus in the State Right-To-Read 
^ * Programs, "Local district directors received from at^least 30 to 
more thaiv200 hours of training. h ■ 

6 Data indicated that States will continue Right-To-^Re^d even in the 
\absence of Federal funds* / \ * */ 

While Right-To-Repd programs have been implemented at^ 
least to a minimal degree across grade levels^ the major 
* « programmatic focus.of RightvTo-Read has b£en ,at the elementary 

Jfevel. . i f 5 
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o # Reading Kas become a top priority in State and local- education 
* , agencies. * , ^ t . 

•* * « * * 

None of the^e outcomes relate directly to the reading achievenry&nt of students, 

because thp -studv #a$, designed solely to assess type and level of technical 

assistance activities (none of which/have ^ direct effect on students) fostered by 

ttle State component ot Right-Tp^Read funding* ^ 

Overcoming Obstacles to Equal Education Opportunity 

Several ED programs are.authorized to help overcome obstacles to learning. 
Among those ^ith recent data aje programs for Native Americans and one for 
preventive education on aldohol and drug abuse; \ * • 

* - . >. > 

Indian Education- , 

Projects to meet the special education needs ^f Indian children are funded under 
Part A of the Indian Education Act. A study of this program found that: 62 . 
percent of these projects focussed on instruction in cultural heritage and native 
language* 58 percent emphasized remedial readings 56 percent emphasized self- 
concept objectives); and 46 percent emphasized remedial mathematics objectives. 
Innovative projects were not always effectively implemented. However, the 
study shows th$t effective implemerttatidh appears to be a function of the ^ 
project's grant size, bbjectives, parent involvement, and Indian population 
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Alcohol and Drug Abuse Education • 
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This program has implegnenterfsa new Strategy which it describes as "teams 
teaching teams." It is w hoped tha^the new strategy will achieve a multiplier 
effect throMgha two-step orpcess* (l) teams are trained/m "clusters" rather than 
in geographically unrelateSgroun?'*' (2) the most impressive trainees in these 



lusters become, through furtheMraining and l^year of onsite technical 
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assistance, qew trainers for other schools and groups of schools in their district 
or community*, while continuing to receive some technical assistance from the^ 
: Federal program. " JL , * 

Two studies funded by the program found thaft (1) trainees felt their training 
was effective; (2) one-half to two-thirds of the teams were still functioning a 
year or two after, training; (3) the majority (about 80 percent) $f the teams were 
performing activities for which they had been trained; and (k) in the judgment of 
the evaluators, thes^, activities were effective. There has been no definitive 
evaluation of impact on student behavior- However, .voluntary reports from 
operating teams cite evidence of substantial decreases in suspensions, vandalism, 
and discipline infractions. Several teams have provided evidence 6f a multiplier 
.effect both by increasing the numbers of teams (and therefore schools covered) 
through training and, in some instances, by raising financial support. Since the 
program does not provide financial support for such reporting, many teams do 
not provide reports and, therefore, it is not claimed that the reporting teams are 
representative of all teams trained. 

* « • 
Learning Resources To Improve the Educational Experience 

. . I 
P± number of programs administered by ED do not "contribute directly to the 

educational process but do develop resources which serve as important adjuncts 

to theprocess 1 . The primary function of most programs in this category is to 

imprOTWibrary services; three bther programs are involved in some aspect of 

educational television. The programs are: 

% Libraries and Learning Resources 

Library Services y * — 
Interhbrary CooperatioiF 

* College Library R<ftour£e*s 
Library Career Training 

Library Researched Demonstration ' * * . 

Strengthening Research Library Resources 
Educational Television and Radio Programming Support * 
Emergency School Aid Act (ESAA)- TV Programming Support 

Library programs provide a variety of support ranging from grants tor 
"hardware 1 ' (e.g., equipment) and "software" (e.g., recorded instructional 
packages) to upgrade the resources of school, college, and public libraries, to* 
training fellowships, and to research and demonstration projects to improve the 
Quality of library practices and services. The programs focused on educational 

.television and radio either support program (series development or improvement 

^of broadcast facilities. 

*> 

ES&A-TV 

# \ -* 
As part of an ongoing study of this program, a sucvey was conducted to 
determine the extent of viewing of TV series produced with ESAA support. The 
survey findings include: 

1 o Availability to Potential Targ^^tudjpnces. For the 11 TV 

Wries available for viewing, the percentage pf potential target 
audiences residing in areas where the series were aired 

* ranged from 3 percent to 81 percent. Four series could be viewed by_ 
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between 32 percent and *2 percent of the target group, and four 
other series couJdbe viewed by 7k percent to 81 percent of the " 
* potential target audiences. 

o Awareness of the TV Series. A relatively smalt, proportion of children 
^ were found to be aware of the TV series. The percentages of 

awareness generally fell in the 20 percent to GO percent range but 
went as low as k percent and a$-high as 55 percent. Overall, less than 
a third of the children had heard of the ESAA series'being shown in . 
their areas. 

it 

o Viewing at Some Time in the Past. Generally, between 10 and 25 

percent of the children. reported having ever seen an ESAA-TV Series. 

q r Viewing Last Week. The proportions on this item wer^low, with 
most ranging between 0 and 5 percent. # 

r 

o Preference oJ^ESAA Series. Children were asked ,to indicate 

their preference between the ESAA series and its competitors in 
the same time slpt. In grades 1 and 4, between 5 and . 
10 percent indicated preference for the ESAA series; in 
grades 7 and 10, between 0 and 5^percent indicated 
sufch a preference. 1 

The viewing data can be interpreted in three ways. The first compares ESAA-TV 
to entertainment television. Given this standard of comparison, the low „ 
viewership of the ESAA-TV series may J>e a cause for concern. A second 
standard of compansor*sees ESAA-TV as educational/public television. The 
viewership findings indicate that ESAA-TV is comparable to other public TV 
programs, which always draw a smaller share of the audience than entertainment 
programs. Finally, ESAA-TV can be perceived as an alternative means to/educe 
cultural isolation. This perspective seeks to exafnine the number of children 
served through various alternative mechanisms) however, the necessary data for 
this compari&n are not presently available. # 

Evaluative studies have recently been completed lor the Library Research and 
Demonstration Program and the Interlibrary Cooperation Program. ( 

The Library Researched Demonstration Program (Title II-B of the Higher 
v> Education Act) has had a significant impact upon the library and information 
community. Title II-B provided millions of dollars for innovative research and 
demonstration; previously few funds were available. This is the only Federal 
program that Supports a wide range of research to improve public library 
services. It has generated new knowledge about innovative approaches to 
networking, serving - special target groups, using community resources, and new 
technologies. Several projects have been successful locally. A few have had ~* 
national impact, such as the series of projects which contributed to the 
development and improvement of the Ohio College Library Center. 
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Program weaknesses include lack of adequate national dissemination, failure to 
adopt successful practices** and lack of a nation|jfjprogram strategy. * 
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The Interlibrary Cooperation Program, Title HI of the^tibrary Services and 
Construction Act (LSCA), has influenced the development^nd expansioi^of 
library cooperation in a number of ways* It is a major force behind the 
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development of multi-library cooperation andj^fc&prking, primarily at the State 
level* Along with Title I of this Act (Library Services Program), Title III has 
resulted m greater centralization ol planning and administering library services 
at State and regional levels* It was also credited as a major influence on State 
legislatures to modify -or pass legislation favoring cooperation and networking. 
Activities and services to increase access to library resources ajid toproVtde 
librarians with needed continuing education were listed <as the major outcomes of 
the. cooperative and networking projects, x 

Although Title III has had a definite, positive impact on the development of 
multi-library (^operation and networking, severed shortcomings and anomalies 
weretdentifitd. First, significant amounts of «LSCA Title I funds (approximately 
Sercent of the fiscal year 1976 appropriation) were used by the States to 
ually or completely suppof t multilibrary cooperation and networking 
activities. It is not known whether this use is detrimental to the mandate of 
Title I (development and support of States efforts to expand and extend public 

iry services to toe unserved and underserved), or if this is an effective means 
of supporting these Types of projects. 

Second, LSCA Title III was providing support. fqr projects that had Betome x 
^ integrated into the States* basic programs of library services. Thislindtfig should 
not be interpreted as indicating that the States are misusing Title III funds. The 
legislatioi^clearly specified that Title III funds can be used to support 
cooperatives and networks. However, given the modest funding level of Title III 
and thevlevel of use of LSCA Title I and State funds,, a shift In emphasis may be 
considered. Slightly, more than 85 percent of the projects receiving Title III 
funds were ongoing projects, scheduled tp continue operartibns for an indefinite 
period of time. Only 1* percent of the projects were short-term, trial projects* 
These figures suggest that Title III funds are targeted more at operational 
.activities than at riskier research and development activities! 

Further, the range of copperative and networking services supported by Title III 
projects was restricted. The interlibrary loan and reference ancj referral 
services were probably efficient and a proper means of gaining access to a wide 
variety of libraries. But there was some question as to how many patrons 
actually needed and used these services; and there jverfe a number of other . , t 
essential library sujJfrort sbrvjces (e.g., document access and sharing, 
warehousing and Shared^torage of low-use materials) which wer.e not being 
provided. The question raised is: How more Title III funds can be directed 
toward exploring new forms of cooperation and networking which will further 
increase the eff icienc/" ai\d economy of library services, as well ^s increase 
access to library resources. r . . 
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the 5 million children who scored in the bottom quarter, only 32 , 
percent (1.6 million) participated in Title I. 

^For these and other teasons, State allocations of Title I migrant funds are 
less than what they might be if all migrant children were identified properly. 

^ + 3 Fully funded in this sepse means that allocations to States are a product 
of the number of eligible children and the authorized per pupil expenditure, and . 
the grants are computed and fOnded "ofNthe-top;" that is, even w£en the Title I_ 
total appropriation falls short oi the authorized level, grants in these two 
programs remain undiminished. Grants to schoil districts under Part A are 
reduced as necessary. / 

• 

* A recent review of educational evaluation recommended greater 
attenjion to measuring implementation. See Robert F. Boruch and David S.> * 
Cordray, A Comprehensive Study of Evaluation Practices and Procedures in 
Federally Funded Elementary and Secondary Education Programs. Study funded 
by the U.S. Office of Education, 1980. 
«. 

'College enrollment rates over time for various subpopulations (defined by 
characteristics such as family income, sex, race, and ethnicity) indicate changes 
in students' pursuit, of poStsecondary school opportunities. However, enrollment 
rates by themselves Say nothing about how overall opportunities for coilege-age 
youth have been changing. For example, any potential increase in enrollment 
figures, for low-income or minority students by increasing available financial aid 
may be offset entirely or partially by improvements in employment prospects for 
noncollege-educated individuals. Such a phenomenon would be most noticeable 
for students whose performance in college Is predicted to be "marginal*" Tables 
presented in this section should be interpreted with this in mind. 

" J ^ 

*>The $L.9 billion is the Federal subsidy for interest, defaults, 
administrative costs, and capital contributions for these programs. However, the 
actual value o\ loans made under the program totaled to $3.5 billion. 

7 The State Student Incentive Grants (SSIG) Program is not included in this 
discussion, since it provides matching grants to States to be used in their own 
student-aid programs and, as such, cannot be differentiated from State funds by 
recipients or financial aid*administrators* • 

V . 

Sit should also' be remembered that there are many students above the 
mean. Thus, the mearv is ortly a broad measure of impact and should be thought 
of as such. 

9 Each Qjjt entry indicates the pr,ecentage of aid applicants in the cell 
whose student burden exceeds the amount indicated at the left — $1,350, $1,850, 
and $2,500* r 

. 

l^See Annual Evaluation Report on Programs Administered by the U.S. 
Office of Education: Fiscal Year 1979 , U.S< Office of Education HE^ t P> 256. 

• * 

' t 1 Significant at less than .01 using a chi-sqiiare test. > 
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l^Thts amount includes funds for the Special Services Program for 
Disadvantaged Students and State Grant Programs. 

^ThiS^tatement should be qualified in.that the students may find other 
opportunities ohequal or higher quality. „ _ * 

'"Higher Education Research Institute; The Impact of Student Financial 
' Aid Programs on Student Choice, Draft Final Report, 1978* 

"Local and State studies are flawed because of insufficient followup and 
inability to control local economic fluctuations and mobilityjof graduates* 

geographically related cluster is comprised of a team from a high 
school joined with teams from the elementary, middle, and/or junior high schools 
feeding into Wat high school. - % 
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This chapter highlights Important findings from recent evaluation studies and 
describes the specific uses of evaluation results. It outlines in broad perspective 
the use of results from ED evaluation studies. Detailed information is included 
in the individual program descriptions in volume II. / 

For each major study, the Office of Evaluation and Program Management 
prepares a 6- to fO-page Executive Summary which briefly describes the" m ^ 
background, methodology, and findings of the study. Persons interested in 
greater detail \bout a study than can be presented in this Chapter should call or 
Write for a copy of the Executive Summary, as follows; " . 

Ms. Yvonne Briscoe 
. Office of Evaluation and Program Management 
1 f JRoom 4070, Switzer Building 

U.S. Department of Education 
Washington, DC 202O2 
- • (202) 2*5-0361 4 , 

<*■ 

Evaluations attest to program successes and f^lures, strengths and weaknesses, 
and thus provide the primary source of objective evidence used in determining 
future program operation and policy options. To be useful, However, evaluation 
and management study findings must be.available when nefeded by Department 
officials for budgets, planning," and operationaCprogram decisions and by the 
Congress for appropriations and reauthorization decisions. As outlined in 
chapter 1, the Office of Evaluation and Program Management has concentrated 
on evaluation followup to promote findings that assist in important 
decisionmaking. Executive Summaries, staff briefings, the Program Resources 
Center, and Evaluation Implementation Boards are all mechanisms for 
incorporating evaluation findings into the decisionmaking process. 

The program evaluation findings described in this chapter can be categorized in 
four areas, basjpd on their use for policy officials and program managers; 
legislation; program eff ectivenesS; program management, and program content. 

* Study of Evaluation Practices and Procedures (Legislation) 

• • 
The recently completed "Study of Evaluation Practices and Procedures" by 
Boruchrand Cordray of Northwestern University recommends that policymakers 
in both the executive an<J legislative branches take a variety of measures in 
mandating and conducting evaluations. In response to the "Holtzman mandate" 
in the Education Amendments of 1978 (Section 1526 of Pubhc°La^95-561) f the 

* report recommends that the Congress: " ; 

o Clarify statutory provisions for evaluation and participate in regular, 
joint planning for. evaluations with the Department of Education. 

o . Conduct a "capability assessment" of grantees' abilities before 
requiring new evaluation tasks. 

o Provide for pilot tests of new program variations in which high . 
quality evaluation designs such as randomized studies can be used. 



o Encourage by law independent analysis and critique of evaluations. 
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, Refer more often to evaluations used in deliberations so Jhat uses of 
evaluations can more easily be tracked* 

o Refrain from incorporating specific evaluation guidelines into law. 

Similar recommendations were made to the Department of Education. The 
report further advised Federal evaluators to consult with congressional offices 
more regularly, make evaluation reports more quickly and easily available, and 
include nftore systematic measurement of program implementation iQ^evaluation 
reports. 

Program Consolidation and the State Role of ESEA-Title IV (Legislation) 

[ \ recent study of the ESEA-Title IV consolidation effort indicated that most 

States and local school districts manage Parts B and C of Title IV independently, 
as they did for the categorical programs subsumed by Title IV. In Title IV, the * 
Congress attempted to consolidate programs that under previous legislation and 
organization were not procedurally or substantively compatible. The absence of 
administrative consolidation at the Federal level apparently influenced States \n 
their administrative decisions. .A related lesson is that just as the Federal 
^ respdnse must represent substantive change, so must a consolidated policy 
represent more than a shuffling together of categorical legislation. 
Further mpre, the Title IV legislatioam the Education Amendments of 1978 
contained no new language to suggest congressional commitment to a • 
consolidated program effort, nor did it include incentives to encourage the 
institutional changes assumed by consolidation. 

Validation of the Student Counts Used To Allocate Funds in Migrant Education 
. (Program Management) " 
. 

Information from a recently completed study, of the 1977 counts used to allocate 
fiscal year 1979 funds for the Migrant Education Program indicated a 12 percent 
undercount of full-time equivalent tFTE) students enrolled in the Migrant 
Student Record Transfer System (MSRTS) during 1977. The major reason for the 
migrant children undercount appeared to be a failure by school districts to 
submit tomplete enrollment data.to the MSRTS. Roughly 15 to 20 percent 6f the 
eligible migrant children enrolled m school during early 1978 were not enrolled in 
the MSRTS in 1977 and therefore did not accrue FTE credit. About 2 percent of 
the FTE's were generated by ineligible children; another 2 percent of the 1977 
FTE's were generated by eligible children while they were out of the country. 
, Overall, the 1977 counts (both for the Nation as a whole aqd fo{ the geographic 
regions defined for the study) seemed to represent a conservative estimate of 
the total number of migrant children. 

* • • 

Evaluation of ESEA Title I Programs for the Neglected and Delinquent (Program 
Management) : " 

The recently completed National Evaluation of State Programs for 
Institutionalized Neglected or Delinquent Youth underTitle I of the Elementry 
and Secondary Education Act describes the program in terms of student need for 
services and actual benefits from participation in Title t. Overall, there is 
0 evidence of significant need but of irregular service delivery. In addition, there 

is scant^evidence of positive change in either basic skills achievement or student 
^ self-pTeiNception. -~ 



On a positive note, special efforts to define chai^cteristics of successful 
programs, as well as practi£es,tohich impede student progress, resulted in 
information which local and State administrators can use to improve their 
programs. For example, a modest relationship was found between student 
exposure to basic skills instruction and achievement growth} yet over half of the 
class-time scheduled for basic skills instruction was actually used for 
noninstructional activities. Of this amount, more than half resulted from 
student absences often due to assignment^ the institution to other activities. 
Only about 25 percent of the class time which was non-task-related could not be 
controlled by staff; the rest of the time went to activities such as smoking 
breaks, clean-up, tracking missing students, and casual conversations. Another 
practice to avoid was the misuse of audiovisual materials to free teacfters from 
teaching. * 

In settings where teachers or aides provided instruction directly, reading gains 
were greater; the fewer students per teacher and the less time teachers and 
students spent in non-task-related conversations, the greater were math and 
reading gains. 

Characteristics and Contexts of ESAA Basic Human Relations Programs 
(Program Content? * 



A recent study of ESAA-funded human relations programs reported an increase 
in the number of these programs over the last few years. The study also 
"reported that: 

o Compared to all ESAA districts, those with human relations programs 

* tended to be larger, more recently desegregated, and to have a lower 
percentage of minority students in the school population. 

o Two types of human relations services were offered to students: 
services focusing primarily onnntergroup relations, problems, and 
understanding; and services focusing primarily on the special needs of 
minority students, s 
* * 

o Programs with a special needs focus occurred in districts that 

* desegregated early, while more recently desegregated districts were 
likely to have Mtergroup relations programs. 

o Human relations services were also offered to staff and parents. 

Staff services consisted primarily of inservice training sessions, and> 
participation in advispry groups was the most common form of 
service to parents. 

Later phases of this study will examine the effectiveness of these services and 
the cljaracteristics of successful human relations programs. 

ESEA, Title I Early Childhood Education (Program Content) 

A recently released report.of an ongoiog stucty of ctarly childhood Compensatory 
education program ^concluded that «arly childhood Title I programs, both within 
and across grades, have extremely diverse strategics and objectives. All the 
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general educational trends common in early childhood education are evident in 
Title I projects* The report also observed that: 

o Prekyidergarten programs typically address a wide range of goals- 
cognitive, language y emotional, social, gross-motor, psychomotor, 
organizational! and motivational development. 

o The structure of kindergarten Title 1 programs depends^bn the 
availablility of other projects within the school district. 

o Virtually all Title 1 first-grade programs are child-focused, school- 

based, and part of a multigrade-level program in reading, language 

arts, or mathematics. 

* +, ♦ 

i 

b ^Without standards or guidelines on how to conduct required 

evaluations of early childhood programs, districts use many methods 
of varying quality. , * 

o Methods to evaluate parent involvement and the consequences for 
their children need to be developed. • 

The findings of this report are being used to review alternative^tfatfegies for an 

evaluation system for early childhood Title 1 programs. 

t * • t * 

State Student Incentive Grant Program (Program Management/Content) 

This study investigated issues concerning the relationship between State and 
Federal agencies in administering student financial aid, with emphasis on the 
State Student Incentiv&Grant (SSIG) Program. SS1G is a $76.5 million per-year 
program designed to encourage States to develop or expand programs of grant 
assistance to needy undergraduate students. Currently* it is a 5Q-50 cost-sharing 
(State-Federal) program. ' 

» r 

The States were found to be sharply divided in terfas of the amount of funds 
provided for student aid. Sixteen States exactly match SSIG funds with State 
aid, while 13 States contribute more than 10 times the SS1G dollars received. 
Further, the needs analysis procedure used Jo determine student eligibility for 
aid, the average and* maximum awards, and the administrative structure 
responsible for State grant programs, vary greatly from State to State. 

The study also found that, at almost all institutions, the tuition refund policy js 
described in the school's admissions catalog. Twenty-five percent notify 
students of refund policies prior to registration, and another 25 percent gfve 
notification to students upon registration. -The average interval between 
application for and Receipt of a refund is 2h weeks. Finally, 66 percent of the 
public institutions and 60 percent of the private institutions have established 
procedures for students to appeal unfavorable refund decisions. 

Criteria for Recognition of Accrediting and State Approval Agencies (Program 
Management) 

Recognition by the Secretary of Education 6f accrediting and State approval 
agencies is the principal means of establishing eligibility to participate in 
Federal student aid programs by postsecondary education institutions. An 
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evaluation of the criteria and procedures for recognition was recently 
completed. Highlights of the study 'findings are: 

- * 
v o Tlje ED procedures for recognition reliably differentiate ineffective 
* "agencies from effective ones, and these procedures are consistent 
over time in interpreting and applying the criteria. _ m 

o The recognitiojvdecisions over the 10-year period from 1969 to 1978 
have consistently utilized virtually all the established criteria. 
_ . i 

o Agencies denied recognition differ among themselves more widely 
tl\an do the agencies recognized. The major deficiencies of ^ie 
agencies denied recognition involve weak eyaluative procedures, 
questions about the impartiality of their decisions, attf 
nonrepresentative governing boards* Other deficiencies are limited 
staff, budget, and experience* 1 

o When presented descriptions of agencies that had been evaluated and 
eitt^er recognized or denied recognition byOED, a group bf slightly 
morft than 100 persons knowledgeable aboyt accrediting agreed 
substantially with the ED'S decisions. The measure of agreement " 
between the actual decisions and the judges' decisions was .68, where 
1.00 indicates perfect agreement. Agreement was nearly unanimous 
for agencies denied recognition 1 -; The major souhce of disagreement 
involved agencies recognized for a limited period, where -judges 
tended to award recognition for a slightly longer period tha(^ actually 
took place* 

A G*mde To MeasurinR Achievement Level and Program Impact on Achievement s 
in Projects (Program Management/Effectiveness) - I 

* ♦ 
One outcome of a study entitled "An Evaluation of Project Information Packages 
(PJP's) As^Used for the Diffusion of Bilingual Projects" was the preparation of a 
prototype evaluation manual. This rpanual contains specific achievement- 
evaluation guides and worksheets which were developed for use in this study. - 
The manual is intended for local bilingual education project directors and 
evaluators who must work together to plan and implement local evaluations. . 
While it is not a complete guide to>bilingual program evaluation, it focuses on 
two important questions: . * * Jfr i 

o What is the level of student performance relative to national 

jiorrris or comparable groups? 
* * 

,o What is the 'impact of the bilingual program on student achieve* 

ment pompared to results of other local instructional practices, past 
and present? 

This unconventional guide to evaluating bilingual projects deals in~depth with 
selected problems that are either unresolved or widely overlooked in current 
evaluations, and it recommends solutions for evaluators at the local level. 

' Title I Technical Assistance Centers (Program Effectiveness) 

^•le Technical Assistance Centers for Title I of the Elementary and Secondary 
gj^£lucation Act provide evaluation technical assistance for Title I elementary and 
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secondary programs to State education agencies and, through the Jftate agencies, 
to local education agencies. There are 10 Technical Assistance Centers (TACs), 
one for each ED region* Many of the problems faced by the State and local . 
education agencies across the country are the same (e.g., test selection, quality 
cgntrol, implementation of the Title I evaluation models, measuring whether 
gains are sustained, aod use of the test and evaluation information). 

To avoid duplication of effort, development of materials was coordinated across 
the TACs. Thus, materials produced by one TAC are available to all other 
TACs and to State and local agenbte^as well, and IACs are required, whenever 
possible, to" use existing material. 

Bilingual Stddent Placement System (Program Management) 

Developed under contract in 1980 b^ED, the Student Placement System offers 
resources to assist bilingual education projects funded under Title VII of the 
Elementary and Secondary Education AcMn the design and implementation of a 
system fori (1) selecting those students most in need of project participation; (2) 
deciding when a participating student may successfully be transferred to an 
English-language classrooms and (3) providing appropriate followup assistance to 
transferred students. In summer 1980, copies of the student placement system 
were distributed to all Title VH-funded bihnguaUgduCation projects. In fall 1980, 
a series of training sessions was held to train staff of the Bilingual Education 
Ser vice Centers and staff of Evaluation, Dissemination, and Assessment Centers 
and, subsequently, to traifi staff of local projects h<*w to use the Student 
Placement System. 



Program Management Procedures in Campus-Based and Basic Grant Programs 
(Program Management) ^ 1 

The rapid expansion of Federal student financial aid programs has put great 
strain on a management "system" that can at best be described as "varying." * A 
recently completed study identified significant gaps that persist in the quality of 
management practices at the institutional level. Four management procedures, 
in particular, were identified as needing ircjproved administration: 



Clarifying and enhancing the role of the student financial aid office 
in the institutional hierarchy. — ~ 

¥ 

Developing and implementing a student financial £id information 
system, at botfftlhe precollege and college level. / 



o Expanding financial aid counsehng^or spec^l student populations* 

o Clarifying and refining the packaging of student financial aid. 

In addition to identifying problems, the study discusses^related issues in detail 
and offers recommendations to help improve institutional operations. The study ' 
does not attempt to duplicate existing manuals on the subject. It does, however, 
review existing publicatibns on the subject for the reader. It also contains 
program i/npact information. — ~~ 

sr .- 
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Impact of the Middle Income Student Assistance Act (Financial Aid) 

"+ 

A study of MlSAA^s impact showed that high proportions of dependent students 
, from all family Jacome levels were receiving aid, but that the proportionate 
increase was greater among those from families with ipcomgs iircxcess of 
$12,000. As a result, the percent of college c<j?ts met frojn work or loans for 
Students with family incomes in excess of $12,000 .declined significantly, 
regardless df type of institutfpn attended. Family contribution expectations also 
declined, according to a recent study of Federal stucjSnt aid programs. , 

At the same time, little change in the relative distribution of need-based Federal 
student aid awards among 4-year institutions was observed. Two-year colleges 
gained as a group while proprietary institutions, which Serve large numbers of 
low income students, showed a loss. All institutions reported increases in the 
"average student award-ranging from $100 to $300. 

Also reported were the concerns of financial aid officers regarding regulations, 
administrative cosjs^iming of funding, and reporting requfrements. 

^Upward Bound (Program Effectiveness) t • 

- A^ecent study of tKe Upward Bound (UB) .program indicated that: 

** * 
o About 91 percent of the typical UB participants entered some type of 
postsecondary education while about 70 pertent of comparable 

, nonparticipants entered. 

* « r * 

o Of the students who entered postsecondary education, about 73 
•percent of typical UB participants attended a 4-year college or 
university. The comparable rate for nonparticipants was 50 percent. 
About 22 percent of the nonparticipants attended a 
vocatiohal/technical school in comparison to 9 percent of the UB 
participants. 

o Typical UB participants received more financial aid to attend 

postsecondary education than comparable nonparticipants. The UB 

E'zipants received twice as much grant/scholarship funding (i.e., 
S vs. $689 on average) as comparable nonparticipants. Typical 
participants aJSb received larger loans (i.e:f$393 vs. $300) than V 
* comparable nonparticipants. 

o Overall, UB had a -large positive effect on student persistence. 
Across all ty^es of schools, typical UB participants maintain their 
enrollment about one term longer than comparable nonparticipants. N 
After the type of school attended is taken into account, UB 
participants persist sightly longer (i.e., one^fifth of a term) than 
> comparable nonparticipants. ' 

• 

o The grades earned at postsecondary institutions by UB participants 
% i*were roughly the same, or slightly lower, than those earned by 

comparable nonparticipants. At 4-year colleges and universities, 
average UB partidpaptT and comparable nonparticipantsjearned 
grade-|>oint, averages of about 2.0 (equivalent to a "C"). 
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Financial Indebtedness and Postsecdndary Education (Program Content) 

Preliminary finding? from a study of student indebtedness indicate that the 
median student dftbt at the bacculaureate level increases frevn $3,000 to $10,000 
for students attending graduate schooK Highest as a group wfrjre Medical 

# students who entered with a median loan amount .of $5,000 ancksaw it grow, to 
$15,000, while at the Iqw end of the scale the median Indebtedness of education 

, majors grew to $3,000 from a base -of $2,000, The study al^o compares expected 
earnings, by discipline, with the level of student indebtedne 

Sex Equity in Vocational Education Programs (Program Effectiveness) 

. This study was mandated by the Congress to determine to what extent sex 
discrimination and stereotyping in vocational programs had been reduced or 
eliminated* Findings indicate that overt discrimination, such as rules denying 
admission on the basis of sex, has been reduced* However, many practices still 
dis&ourage students from erjtenrtg nontraditional fields before 'they are enrolled 
and further discourage them once they are enpailed. Despite societal factors 
^ fc outside the school which exert powerful, influences on the aspirations of youth, 

^ study findings indicate that the school can have an impact because of the strong 

. positive correlation .between the school's level oi equity activity and its. 

- proportion ot nontraditional enrollment. "Howe\*er, little action was evident*at 

the-.school level to foster equity —particularly student-oriented activity. 

Fey, States or schools were engaged in community and employer activities. Yet 
these activities — particularly with employers — seenrt to be a critical ingredient 
r in any successful effort to promote sex equity. There is also evidence that 

efforts to achieve equity in staffing patterns have a positive effect. Ihpse* 
w ~ schools with higher "internal affirmative action" indexes also had highCT 

proportions of nontraditional teachers. 

National Studfr of Vocational Education System and Facilities (Program Content) 

A comprehensive study of the vocational education systems in each of the States 
^ and territories has provided the first thorough description of State governance 

* structures, delivery systems, and funding arrangements. Prior to this study, 

heterogeneity among the States was known to be characteristic of vocational 

education; the study documents these differences. During the bourse of the 

study, information gaps recognized at the outset were found to be even greater 

tfQn anticipated. ' t * * 

* 

The report deals with the organization, governance, and financings! -the State 

systems; the organization ^t the' local levels within each State; and State 

financial policies as they relate Jo vocational education facilities, equipment, 

and program operations. The number, characteristics, and utilization of the 1 

several types of vocational educational institutions are also assessed for the first 
t * * • 

tim'e. : * * • 1 

The general characteristics of 6,660 institutions (5,560 secondary and'1,100 ^ 
postsecondary) are described. Among these characteristics are the number and 
v geographical distribution of the institutions, shops^nd laboratories; typee of. 
construction of f acilitiesif re porte d condition of facilities, and accessibility to 



.handicapped persons. 
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Other findings show that: » 

o The greatest concentration of vocational education facilities (shops 
and laboratory spaces) is in communities of 25,000 - 100,000 
population. * fc x 

o In 1965, W States financed construction projects with Federal 

vocational education funds; in 1975, only 21 States used these funds. 

o The proportion of vocational education facilities reported as totally 
inaccessible to the handicapped is greatest among vocationa l his, x 
schools; over 30 percent are inaccessible, compared to lesser rates 

* at other other types of vocational education institutions. 

* 

Assessment of State Plans for Career Education (Program Management) 

\*A recently completed study has provided 49 States and Territories with * 
evaluations of their State plans for implementing career education. Initial 
support for developing these fflans was provided during fiscaj years 1976 and 
L977, The Study, which started in September 1978, identified 10 elements of a 
model State plan. Criteria were developed so that each element of a plan could 
be rated" on a three-point scale as being "strong," "adequate," or "weak." An 
. 9 overall rating was then calculated on a five-point scale. 

■ . * 

The 49 States and Territories which submitted plans before May 1979 were given 
preliminary ratings of their plans. In these ratings, 23 (47 percent) received an 
overall rating of 3.0 or higher. In response, over half of the sites (57 percent) 
voluntarily provided revisions and/or additions to remedy reported weaknesses. 
Final rating^ were then sent to all States. This time, 30 (62 percent) were rated 
3.0 ^"adequate") or better. Seven of the plans were rated."strong" and only three 
were considered "weakl" / « 

-* ■ * 

The States and Territories also received a guide to help them adapt their plans to 
meet the requirements of P.L. 95*507, the new Career Education Incentive Act; 
This guide togejher with the assessment under the old law enabled States to 
incorporate the information gained from the individual evaluations of their plans 
into the new 5-year plan required by the Career Education Incentive Act. These 
plans form the basis for funding in fiscal year 1980 and beyond. 

i 

A>tudy of Library Cooperatives, Networks, and Demonstration Projects 
(Program Effectiveness) : * ~ ~ ~ 

A study of the Library Research and Demonstration Program and the Interlibrary 
Cooperation Program found that both programs were making significant 

# contributions to th$ Improvements libraries. , * . 

-* **** v f 

The Research and Demonstration Program has provided millions of dollars for * 
innovative research «frid demonstration, .where either very few or no other funds 
were available. Local projects often focused on providing library service to 
minorities and on the r utilization of new .technologies. » 

* The Interlibrary Cooperation Program is responsible m part for the development 

♦ of multiUbrary cooperation and networking, especially at State and regional 
jewels. It was alsfc found tp be a major influence toward modifying or passing 

FRir te legislation that favored cooperation and networking. 
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This chapter has presented highlights from study findings. Volume II will discuss 
these findings in greater detail, along with program-specific information. 
Volume II describes each program's legislative funding history, objectives, 
pperations, and evaluation findings* v 

A program's funding history in volume II will consist of annual authorizations and 
subsequent appropriations; program goals and objectives state thl broad ^ 
congressional intent in establishing each education program* The program 
operations sections in volume II will discuss how programs actually have been 
implemented and describe the scope of current activities. The evaluation 
findings will focus on program effectiveness and "progress toward achieving long- 
range program goals* 

In subsequent years, volume II will report on program changes in detail* 'Volume 
I, on the other hand, will continue to cover th^ evaluation cole of the 
Departmental Education from a broader perspective* Significant changes in the 
evaluation mission will be explained along with any new analytic approaches 
adopted or established techniques that may be de-emphasized* 




* 
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